LSTMtextGeneration.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def read_file(filepath):\n",
    "    with open(filepath) as f:\n",
    "        str_text = f.read()\n",
    "        \n",
    "    return str_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Call me Ishmael.  Some years ago--never mind how long\\nprecisely--having little or no money in my purse, and nothing\\nparticular to interest me on shore, I thought I would sail about a\\nlittle and see the watery part of the world.  It is a way I have of\\ndriving off the spleen and regulating the circulation.  Whenever I\\nfind myself growing grim about the mouth; whenever it is a damp,\\ndrizzly November in my soul; whenever I find myself involuntarily\\npausing before coffin warehouses, and bringing up the rear of every\\nfuneral I meet; and especially whenever my hypos get such an upper\\nhand of me, that it requires a strong moral principle to prevent me\\nfrom deliberately stepping into the street, and methodically knocking\\npeople\\'s hats off--then, I account it high time to get to sea as soon\\nas I can.  This is my substitute for pistol and ball.  With a\\nphilosophical flourish Cato throws himself upon his sword; I quietly\\ntake to the ship.  There is nothing surprising in this.  If they but\\nknew it, almost all men in their degree, some time or other, cherish\\nvery nearly the same feelings towards the ocean with me.\\n\\nThere now is your insular city of the Manhattoes, belted round by\\nwharves as Indian isles by coral reefs--commerce surrounds it with\\nher surf.  Right and left, the streets take you waterward.  Its\\nextreme downtown is the battery, where that noble mole is washed by\\nwaves, and cooled by breezes, which a few hours previous were out of\\nsight of land.  Look at the crowds of water-gazers there.\\n\\nCircumambulate the city of a dreamy Sabbath afternoon.  Go from\\nCorlears Hook to Coenties Slip, and from thence, by Whitehall,\\nnorthward.  What do you see?--Posted like silent sentinels all around\\nthe town, stand thousands upon thousands of mortal men fixed in ocean\\nreveries.  Some leaning against the spiles; some seated upon the\\npier-heads; some looking over the bulwarks of ships from China; some\\nhigh aloft in the rigging, as if striving to get a still better\\nseaward peep.  But these are all landsmen; of week days pent up in\\nlath and plaster--tied to counters, nailed to benches, clinched to\\ndesks.  How then is this?  Are the green fields gone?  What do they\\nhere?\\n\\nBut look! here come more crowds, pacing straight for the water, and\\nseemingly bound for a dive.  Strange!  Nothing will content them but\\nthe extremest limit of the land; loitering under the shady lee of\\nyonder warehouses will not suffice.  No.  They must get just as nigh\\nthe water as they possibly can without falling in.  And there they\\nstand--miles of them--leagues.  Inlanders all, they come from lanes\\nand alleys, streets and avenues--north, east, south, and west.  Yet\\nhere they all unite.  Tell me, does the magnetic virtue of the\\nneedles of the compasses of all those ships attract them thither?\\n\\nOnce more.  Say you are in the country; in some high land of lakes.\\nTake almost any path you please, and ten to one it carries you down\\nin a dale, and leaves you there by a pool in the stream.  There is\\nmagic in it.  Let the most absent-minded of men be plunged in his\\ndeepest reveries--stand that man on his legs, set his feet a-going,\\nand he will infallibly lead you to water, if water there be in all\\nthat region.  Should you ever be athirst in the great American\\ndesert, try this experiment, if your caravan happen to be supplied\\nwith a metaphysical professor.  Yes, as every one knows, meditation\\nand water are wedded for ever.\\n\\nBut here is an artist.  He desires to paint you the dreamiest,\\nshadiest, quietest, most enchanting bit of romantic landscape in all\\nthe valley of the Saco.  What is the chief element he employs?  There\\nstand his trees, each with a hollow trunk, as if a hermit and a\\ncrucifix were within; and here sleeps his meadow, and there sleep his\\ncattle; and up from yonder cottage goes a sleepy smoke.  Deep into\\ndistant woodlands winds a mazy way, reaching to overlapping spurs of\\nmountains bathed in their hill-side blue.  But though the picture\\nlies thus tranced, and though this pine-tree shakes down its sighs\\nlike leaves upon this shepherd\\'s head, yet all were vain, unless the\\nshepherd\\'s eye were fixed upon the magic stream before him.  Go visit\\nthe Prairies in June, when for scores on scores of miles you wade\\nknee-deep among Tiger-lilies--what is the one charm\\nwanting?--Water--there is not a drop of water there!  Were Niagara\\nbut a cataract of sand, would you travel your thousand miles to see\\nit?  Why did the poor poet of Tennessee, upon suddenly receiving two\\nhandfuls of silver, deliberate whether to buy him a coat, which he\\nsadly needed, or invest his money in a pedestrian trip to Rockaway\\nBeach?  Why is almost every robust healthy boy with a robust healthy\\nsoul in him, at some time or other crazy to go to sea?  Why upon your\\nfirst voyage as a passenger, did you yourself feel such a mystical\\nvibration, when first told that you and your ship were now out of\\nsight of land?  Why did the old Persians hold the sea holy?  Why did\\nthe Greeks give it a separate deity, and own brother of Jove?  Surely\\nall this is not without meaning.  And still deeper the meaning of\\nthat story of Narcissus, who because he could not grasp the\\ntormenting, mild image he saw in the fountain, plunged into it and\\nwas drowned.  But that same image, we ourselves see in all rivers and\\noceans.  It is the image of the ungraspable phantom of life; and this\\nis the key to it all.\\n\\nNow, when I say that I am in the habit of going to sea whenever I\\nbegin to grow hazy about the eyes, and begin to be over conscious of\\nmy lungs, I do not mean to have it inferred that I ever go to sea as\\na passenger.  For to go as a passenger you must needs have a purse,\\nand a purse is but a rag unless you have something in it.  Besides,\\npassengers get sea-sick--grow quarrelsome--don\\'t sleep of nights--do\\nnot enjoy themselves much, as a general thing;--no, I never go as a\\npassenger; nor, though I am something of a salt, do I ever go to sea\\nas a Commodore, or a Captain, or a Cook.  I abandon the glory and\\ndistinction of such offices to those who like them.  For my part, I\\nabominate all honourable respectable toils, trials, and tribulations\\nof every kind whatsoever.  It is quite as much as I can do to take\\ncare of myself, without taking care of ships, barques, brigs,\\nschooners, and what not.  And as for going as cook,--though I confess\\nthere is considerable glory in that, a cook being a sort of officer\\non ship-board--yet, somehow, I never fancied broiling fowls;--though\\nonce broiled, judiciously buttered, and judgmatically salted and\\npeppered, there is no one who will speak more respectfully, not to\\nsay reverentially, of a broiled fowl than I will.  It is out of the\\nidolatrous dotings of the old Egyptians upon broiled ibis and roasted\\nriver horse, that you see the mummies of those creatures in their\\nhuge bake-houses the pyramids.\\n\\nNo, when I go to sea, I go as a simple sailor, right before the mast,\\nplumb down into the forecastle, aloft there to the royal mast-head.\\nTrue, they rather order me about some, and make me jump from spar to\\nspar, like a grasshopper in a May meadow.  And at first, this sort of\\nthing is unpleasant enough.  It touches one\\'s sense of honour,\\nparticularly if you come of an old established family in the land,\\nthe Van Rensselaers, or Randolphs, or Hardicanutes.  And more than\\nall, if just previous to putting your hand into the tar-pot, you have\\nbeen lording it as a country schoolmaster, making the tallest boys\\nstand in awe of you.  The transition is a keen one, I assure you,\\nfrom a schoolmaster to a sailor, and requires a strong decoction of\\nSeneca and the Stoics to enable you to grin and bear it.  But even\\nthis wears off in time.\\n\\nWhat of it, if some old hunks of a sea-captain orders me to get a\\nbroom and sweep down the decks?  What does that indignity amount to,\\nweighed, I mean, in the scales of the New Testament?  Do you think\\nthe archangel Gabriel thinks anything the less of me, because I\\npromptly and respectfully obey that old hunks in that particular\\ninstance?  Who ain\\'t a slave?  Tell me that.  Well, then, however the\\nold sea-captains may order me about--however they may thump and punch\\nme about, I have the satisfaction of knowing that it is all right;\\nthat everybody else is one way or other served in much the same\\nway--either in a physical or metaphysical point of view, that is; and\\nso the universal thump is passed round, and all hands should rub each\\nother\\'s shoulder-blades, and be content.\\n\\nAgain, I always go to sea as a sailor, because they make a point of\\npaying me for my trouble, whereas they never pay passengers a single\\npenny that I ever heard of.  On the contrary, passengers themselves\\nmust pay.  And there is all the difference in the world between\\npaying and being paid.  The act of paying is perhaps the most\\nuncomfortable infliction that the two orchard thieves entailed upon\\nus.  But BEING PAID,--what will compare with it?  The urbane activity\\nwith which a man receives money is really marvellous, considering\\nthat we so earnestly believe money to be the root of all earthly\\nills, and that on no account can a monied man enter heaven.  Ah! how\\ncheerfully we consign ourselves to perdition!\\n\\nFinally, I always go to sea as a sailor, because of the wholesome\\nexercise and pure air of the fore-castle deck.  For as in this world,\\nhead winds are far more prevalent than winds from astern (that is, if\\nyou never violate the Pythagorean maxim), so for the most part the\\nCommodore on the quarter-deck gets his atmosphere at second hand from\\nthe sailors on the forecastle.  He thinks he breathes it first; but\\nnot so.  In much the same way do the commonalty lead their leaders in\\nmany other things, at the same time that the leaders little suspect\\nit.  But wherefore it was that after having repeatedly smelt the sea\\nas a merchant sailor, I should now take it into my head to go on a\\nwhaling voyage; this the invisible police officer of the Fates, who\\nhas the constant surveillance of me, and secretly dogs me, and\\ninfluences me in some unaccountable way--he can better answer than\\nany one else.  And, doubtless, my going on this whaling voyage,\\nformed part of the grand programme of Providence that was drawn up a\\nlong time ago.  It came in as a sort of brief interlude and solo\\nbetween more extensive performances.  I take it that this part of the\\nbill must have run something like this:\\n\\n\\n\"GRAND CONTESTED ELECTION FOR THE PRESIDENCY OF THE UNITED STATES.\\n\"WHALING VOYAGE BY ONE ISHMAEL.\\n\"BLOODY BATTLE IN AFFGHANISTAN.\"\\n\\n\\nThough I cannot tell why it was exactly that those stage managers,\\nthe Fates, put me down for this shabby part of a whaling voyage, when\\nothers were set down for magnificent parts in high tragedies, and\\nshort and easy parts in genteel comedies, and jolly parts in\\nfarces--though I cannot tell why this was exactly; yet, now that I\\nrecall all the circumstances, I think I can see a little into the\\nsprings and motives which being cunningly presented to me under\\nvarious disguises, induced me to set about performing the part I did,\\nbesides cajoling me into the delusion that it was a choice resulting\\nfrom my own unbiased freewill and discriminating judgment.\\n\\nChief among these motives was the overwhelming idea of the great\\nwhale himself.  Such a portentous and mysterious monster roused all\\nmy curiosity.  Then the wild and distant seas where he rolled his\\nisland bulk; the undeliverable, nameless perils of the whale; these,\\nwith all the attending marvels of a thousand Patagonian sights and\\nsounds, helped to sway me to my wish.  With other men, perhaps, such\\nthings would not have been inducements; but as for me, I am tormented\\nwith an everlasting itch for things remote.  I love to sail forbidden\\nseas, and land on barbarous coasts.  Not ignoring what is good, I am\\nquick to perceive a horror, and could still be social with it--would\\nthey let me--since it is but well to be on friendly terms with all\\nthe inmates of the place one lodges in.\\n\\nBy reason of these things, then, the whaling voyage was welcome; the\\ngreat flood-gates of the wonder-world swung open, and in the wild\\nconceits that swayed me to my purpose, two and two there floated into\\nmy inmost soul, endless processions of the whale, and, mid most of\\nthem all, one grand hooded phantom, like a snow hill in the air.\\n\\nI stuffed a shirt or two into my old carpet-bag, tucked it under my\\narm, and started for Cape Horn and the Pacific.  Quitting the good\\ncity of old Manhatto, I duly arrived in New Bedford.  It was a\\nSaturday night in December.  Much was I disappointed upon learning\\nthat the little packet for Nantucket had already sailed, and that no\\nway of reaching that place would offer, till the following Monday.\\n\\nAs most young candidates for the pains and penalties of whaling stop\\nat this same New Bedford, thence to embark on their voyage, it may as\\nwell be related that I, for one, had no idea of so doing.  For my\\nmind was made up to sail in no other than a Nantucket craft, because\\nthere was a fine, boisterous something about everything connected\\nwith that famous old island, which amazingly pleased me.  Besides\\nthough New Bedford has of late been gradually monopolising the\\nbusiness of whaling, and though in this matter poor old Nantucket is\\nnow much behind her, yet Nantucket was her great original--the Tyre\\nof this Carthage;--the place where the first dead American whale was\\nstranded.  Where else but from Nantucket did those aboriginal\\nwhalemen, the Red-Men, first sally out in canoes to give chase to the\\nLeviathan?  And where but from Nantucket, too, did that first\\nadventurous little sloop put forth, partly laden with imported\\ncobblestones--so goes the story--to throw at the whales, in order to\\ndiscover when they were nigh enough to risk a harpoon from the\\nbowsprit?\\n\\nNow having a night, a day, and still another night following before\\nme in New Bedford, ere I could embark for my destined port, it\\nbecame a matter of concernment where I was to eat and sleep\\nmeanwhile.  It was a very dubious-looking, nay, a very dark and\\ndismal night, bitingly cold and cheerless.  I knew no one in the\\nplace.  With anxious grapnels I had sounded my pocket, and only\\nbrought up a few pieces of silver,--So, wherever you go, Ishmael,\\nsaid I to myself, as I stood in the middle of a dreary street\\nshouldering my bag, and comparing the gloom towards the north with\\nthe darkness towards the south--wherever in your wisdom you may\\nconclude to lodge for the night, my dear Ishmael, be sure to inquire\\nthe price, and don\\'t be too particular.\\n\\nWith halting steps I paced the streets, and passed the sign of \"The\\nCrossed Harpoons\"--but it looked too expensive and jolly there.\\nFurther on, from the bright red windows of the \"Sword-Fish Inn,\"\\nthere came such fervent rays, that it seemed to have melted the\\npacked snow and ice from before the house, for everywhere else the\\ncongealed frost lay ten inches thick in a hard, asphaltic\\npavement,--rather weary for me, when I struck my foot against the\\nflinty projections, because from hard, remorseless service the soles\\nof my boots were in a most miserable plight.  Too expensive and\\njolly, again thought I, pausing one moment to watch the broad glare\\nin the street, and hear the sounds of the tinkling glasses within.\\nBut go on, Ishmael, said I at last; don\\'t you hear? get away from\\nbefore the door; your patched boots are stopping the way.  So on I\\nwent.  I now by instinct followed the streets that took me waterward,\\nfor there, doubtless, were the cheapest, if not the cheeriest inns.\\n\\nSuch dreary streets! blocks of blackness, not houses, on either\\nhand, and here and there a candle, like a candle moving about in a\\ntomb.  At this hour of the night, of the last day of the week, that\\nquarter of the town proved all but deserted.  But presently I came to\\na smoky light proceeding from a low, wide building, the door of which\\nstood invitingly open.  It had a careless look, as if it were meant\\nfor the uses of the public; so, entering, the first thing I did was\\nto stumble over an ash-box in the porch.  Ha! thought I, ha, as the\\nflying particles almost choked me, are these ashes from that\\ndestroyed city, Gomorrah?  But \"The Crossed Harpoons,\" and \"The\\nSword-Fish?\"--this, then must needs be the sign of \"The Trap.\"\\nHowever, I picked myself up and hearing a loud voice within, pushed\\non and opened a second, interior door.\\n\\nIt seemed the great Black Parliament sitting in Tophet.  A hundred\\nblack faces turned round in their rows to peer; and beyond, a black\\nAngel of Doom was beating a book in a pulpit.  It was a negro church;\\nand the preacher\\'s text was about the blackness of darkness, and the\\nweeping and wailing and teeth-gnashing there.  Ha, Ishmael, muttered\\nI, backing out, Wretched entertainment at the sign of \\'The Trap!\\'\\n\\nMoving on, I at last came to a dim sort of light not far from the\\ndocks, and heard a forlorn creaking in the air; and looking up, saw a\\nswinging sign over the door with a white painting upon it, faintly\\nrepresenting a tall straight jet of misty spray, and these words\\nunderneath--\"The Spouter Inn:--Peter Coffin.\"\\n\\nCoffin?--Spouter?--Rather ominous in that particular connexion,\\nthought I.  But it is a common name in Nantucket, they say, and I\\nsuppose this Peter here is an emigrant from there.  As the light\\nlooked so dim, and the place, for the time, looked quiet enough, and\\nthe dilapidated little wooden house itself looked as if it might have\\nbeen carted here from the ruins of some burnt district, and as the\\nswinging sign had a poverty-stricken sort of creak to it, I thought\\nthat here was the very spot for cheap lodgings, and the best of pea\\ncoffee.\\n\\nIt was a queer sort of place--a gable-ended old house, one side\\npalsied as it were, and leaning over sadly.  It stood on a sharp\\nbleak corner, where that tempestuous wind Euroclydon kept up a worse\\nhowling than ever it did about poor Paul\\'s tossed craft.  Euroclydon,\\nnevertheless, is a mighty pleasant zephyr to any one in-doors, with\\nhis feet on the hob quietly toasting for bed.  \"In judging of that\\ntempestuous wind called Euroclydon,\" says an old writer--of whose\\nworks I possess the only copy extant--\"it maketh a marvellous\\ndifference, whether thou lookest out at it from a glass window where\\nthe frost is all on the outside, or whether thou observest it from\\nthat sashless window, where the frost is on both sides, and of which\\nthe wight Death is the only glazier.\"  True enough, thought I, as\\nthis passage occurred to my mind--old black-letter, thou reasonest\\nwell.  Yes, these eyes are windows, and this body of mine is the\\nhouse.  What a pity they didn\\'t stop up the chinks and the crannies\\nthough, and thrust in a little lint here and there.  But it\\'s too\\nlate to make any improvements now.  The universe is finished; the\\ncopestone is on, and the chips were carted off a million years ago.\\nPoor Lazarus there, chattering his teeth against the curbstone for\\nhis pillow, and shaking off his tatters with his shiverings, he might\\nplug up both ears with rags, and put a corn-cob into his mouth, and\\nyet that would not keep out the tempestuous Euroclydon.  Euroclydon!\\nsays old Dives, in his red silken wrapper--(he had a redder one\\nafterwards) pooh, pooh!  What a fine frosty night; how Orion\\nglitters; what northern lights!  Let them talk of their oriental\\nsummer climes of everlasting conservatories; give me the privilege of\\nmaking my own summer with my own coals.\\n\\nBut what thinks Lazarus?  Can he warm his blue hands by holding them\\nup to the grand northern lights?  Would not Lazarus rather be in\\nSumatra than here?  Would he not far rather lay him down lengthwise\\nalong the line of the equator; yea, ye gods! go down to the fiery pit\\nitself, in order to keep out this frost?\\n\\nNow, that Lazarus should lie stranded there on the curbstone before\\nthe door of Dives, this is more wonderful than that an iceberg should\\nbe moored to one of the Moluccas.  Yet Dives himself, he too lives\\nlike a Czar in an ice palace made of frozen sighs, and being a\\npresident of a temperance society, he only drinks the tepid tears of\\norphans.\\n\\nBut no more of this blubbering now, we are going a-whaling, and there\\nis plenty of that yet to come.  Let us scrape the ice from our\\nfrosted feet, and see what sort of a place this \"Spouter\" may be.\\n\\nEntering that gable-ended Spouter-Inn, you found yourself in a wide,\\nlow, straggling entry with old-fashioned wainscots, reminding one of\\nthe bulwarks of some condemned old craft.  On one side hung a very\\nlarge oilpainting so thoroughly besmoked, and every way defaced,\\nthat in the unequal crosslights by which you viewed it, it was only\\nby diligent study and a series of systematic visits to it, and\\ncareful inquiry of the neighbors, that you could any way arrive at an\\nunderstanding of its purpose.  Such unaccountable masses of shades\\nand shadows, that at first you almost thought some ambitious young\\nartist, in the time of the New England hags, had endeavored to\\ndelineate chaos bewitched.  But by dint of much and earnest\\ncontemplation, and oft repeated ponderings, and especially by\\nthrowing open the little window towards the back of the entry, you at\\nlast come to the conclusion that such an idea, however wild, might\\nnot be altogether unwarranted.\\n\\nBut what most puzzled and confounded you was a long, limber,\\nportentous, black mass of something hovering in the centre of the\\npicture over three blue, dim, perpendicular lines floating in a\\nnameless yeast.  A boggy, soggy, squitchy picture truly, enough to\\ndrive a nervous man distracted.  Yet was there a sort of indefinite,\\nhalf-attained, unimaginable sublimity about it that fairly froze you\\nto it, till you involuntarily took an oath with yourself to find out\\nwhat that marvellous painting meant.  Ever and anon a bright, but,\\nalas, deceptive idea would dart you through.--It\\'s the Black Sea in a\\nmidnight gale.--It\\'s the unnatural combat of the four primal\\nelements.--It\\'s a blasted heath.--It\\'s a Hyperborean winter\\nscene.--It\\'s the breaking-up of the icebound stream of Time.  But at\\nlast all these fancies yielded to that one portentous something in\\nthe picture\\'s midst.  THAT once found out, and all the rest were\\nplain.  But stop; does it not bear a faint resemblance to a gigantic\\nfish? even the great leviathan himself?\\n\\nIn fact, the artist\\'s design seemed this: a final theory of my own,\\npartly based upon the aggregated opinions of many aged persons with\\nwhom I conversed upon the subject.  The picture represents a\\nCape-Horner in a great hurricane; the half-foundered ship weltering\\nthere with its three dismantled masts alone visible; and an\\nexasperated whale, purposing to spring clean over the craft, is in\\nthe enormous act of impaling himself upon the three mast-heads.\\n\\nThe opposite wall of this entry was hung all over with a heathenish\\narray of monstrous clubs and spears.  Some were thickly set with\\nglittering teeth resembling ivory saws; others were tufted with knots\\nof human hair; and one was sickle-shaped, with a vast handle sweeping\\nround like the segment made in the new-mown grass by a long-armed\\nmower.  You shuddered as you gazed, and wondered what monstrous\\ncannibal and savage could ever have gone a death-harvesting with such\\na hacking, horrifying implement.  Mixed with these were rusty old\\nwhaling lances and harpoons all broken and deformed.  Some were\\nstoried weapons.  With this once long lance, now wildly elbowed,\\nfifty years ago did Nathan Swain kill fifteen whales between a\\nsunrise and a sunset.  And that harpoon--so like a corkscrew now--was\\nflung in Javan seas, and run away with by a whale, years afterwards\\nslain off the Cape of Blanco.  The original iron entered nigh the\\ntail, and, like a restless needle sojourning in the body of a man,\\ntravelled full forty feet, and at last was found imbedded in the\\nhump.\\n\\nCrossing this dusky entry, and on through yon low-arched way--cut\\nthrough what in old times must have been a great central chimney with\\nfireplaces all round--you enter the public room.  A still duskier\\nplace is this, with such low ponderous beams above, and such old\\nwrinkled planks beneath, that you would almost fancy you trod some\\nold craft\\'s cockpits, especially of such a howling night, when this\\ncorner-anchored old ark rocked so furiously.  On one side stood a\\nlong, low, shelf-like table covered with cracked glass cases, filled\\nwith dusty rarities gathered from this wide world\\'s remotest nooks.\\nProjecting from the further angle of the room stands a dark-looking\\nden--the bar--a rude attempt at a right whale\\'s head.  Be that how it\\nmay, there stands the vast arched bone of the whale\\'s jaw, so wide, a\\ncoach might almost drive beneath it.  Within are shabby shelves,\\nranged round with old decanters, bottles, flasks; and in those jaws\\nof swift destruction, like another cursed Jonah (by which name indeed\\nthey called him), bustles a little withered old man, who, for their\\nmoney, dearly sells the sailors deliriums and death.\\n\\nAbominable are the tumblers into which he pours his poison.  Though\\ntrue cylinders without--within, the villanous green goggling glasses\\ndeceitfully tapered downwards to a cheating bottom.  Parallel\\nmeridians rudely pecked into the glass, surround these footpads\\'\\ngoblets.  Fill to THIS mark, and your charge is but a penny; to THIS\\na penny more; and so on to the full glass--the Cape Horn measure,\\nwhich you may gulp down for a shilling.\\n\\nUpon entering the place I found a number of young seamen gathered\\nabout a table, examining by a dim light divers specimens of\\nSKRIMSHANDER.  I sought the landlord, and telling him I desired to be\\naccommodated with a room, received for answer that his house was\\nfull--not a bed unoccupied.  \"But avast,\" he added, tapping his\\nforehead, \"you haint no objections to sharing a harpooneer\\'s blanket,\\nhave ye?  I s\\'pose you are goin\\' a-whalin\\', so you\\'d better get used\\nto that sort of thing.\"\\n\\nI told him that I never liked to sleep two in a bed; that if I should\\never do so, it would depend upon who the harpooneer might be, and\\nthat if he (the landlord) really had no other place for me, and the\\nharpooneer was not decidedly objectionable, why rather than wander\\nfurther about a strange town on so bitter a night, I would put up\\nwith the half of any decent man\\'s blanket.\\n\\n\"I thought so.  All right; take a seat.  Supper?--you want supper?\\nSupper\\'ll be ready directly.\"\\n\\nI sat down on an old wooden settle, carved all over like a bench on\\nthe Battery.  At one end a ruminating tar was still further adorning\\nit with his jack-knife, stooping over and diligently working away at\\nthe space between his legs.  He was trying his hand at a ship under\\nfull sail, but he didn\\'t make much headway, I thought.\\n\\nAt last some four or five of us were summoned to our meal in an\\nadjoining room.  It was cold as Iceland--no fire at all--the landlord\\nsaid he couldn\\'t afford it.  Nothing but two dismal tallow candles,\\neach in a winding sheet.  We were fain to button up our monkey\\njackets, and hold to our lips cups of scalding tea with our half\\nfrozen fingers.  But the fare was of the most substantial kind--not\\nonly meat and potatoes, but dumplings; good heavens! dumplings for\\nsupper!  One young fellow in a green box coat, addressed himself to\\nthese dumplings in a most direful manner.\\n\\n\"My boy,\" said the landlord, \"you\\'ll have the nightmare to a dead\\nsartainty.\"\\n\\n\"Landlord,\" I whispered, \"that aint the harpooneer is it?\"\\n\\n\"Oh, no,\" said he, looking a sort of diabolically funny, \"the\\nharpooneer is a dark complexioned chap.  He never eats dumplings, he\\ndon\\'t--he eats nothing but steaks, and he likes \\'em rare.\"\\n\\n\"The devil he does,\" says I.  \"Where is that harpooneer?  Is he\\nhere?\"\\n\\n\"He\\'ll be here afore long,\" was the answer.\\n\\nI could not help it, but I began to feel suspicious of this \"dark\\ncomplexioned\" harpooneer.  At any rate, I made up my mind that if it\\nso turned out that we should sleep together, he must undress and get\\ninto bed before I did.\\n\\nSupper over, the company went back to the bar-room, when, knowing not\\nwhat else to do with myself, I resolved to spend the rest of the\\nevening as a looker on.\\n\\nPresently a rioting noise was heard without.  Starting up, the\\nlandlord cried, \"That\\'s the Grampus\\'s crew.  I seed her reported in\\nthe offing this morning; a three years\\' voyage, and a full ship.\\nHurrah, boys; now we\\'ll have the latest news from the Feegees.\"\\n\\nA tramping of sea boots was heard in the entry; the door was flung\\nopen, and in rolled a wild set of mariners enough.  Enveloped in\\ntheir shaggy watch coats, and with their heads muffled in woollen\\ncomforters, all bedarned and ragged, and their beards stiff with\\nicicles, they seemed an eruption of bears from Labrador.  They had\\njust landed from their boat, and this was the first house they\\nentered.  No wonder, then, that they made a straight wake for the\\nwhale\\'s mouth--the bar--when the wrinkled little old Jonah, there\\nofficiating, soon poured them out brimmers all round.  One complained\\nof a bad cold in his head, upon which Jonah mixed him a pitch-like\\npotion of gin and molasses, which he swore was a sovereign cure for\\nall colds and catarrhs whatsoever, never mind of how long standing,\\nor whether caught off the coast of Labrador, or on the weather side\\nof an ice-island.\\n\\nThe liquor soon mounted into their heads, as it generally does even\\nwith the arrantest topers newly landed from sea, and they began\\ncapering about most obstreperously.\\n\\nI observed, however, that one of them held somewhat aloof, and though\\nhe seemed desirous not to spoil the hilarity of his shipmates by his\\nown sober face, yet upon the whole he refrained from making as much\\nnoise as the rest.  This man interested me at once; and since the\\nsea-gods had ordained that he should soon become my shipmate (though\\nbut a sleeping-partner one, so far as this narrative is concerned),\\nI will here venture upon a little description of him.  He stood full\\nsix feet in height, with noble shoulders, and a chest like a\\ncoffer-dam.  I have seldom seen such brawn in a man.  His face was\\ndeeply brown and burnt, making his white teeth dazzling by the\\ncontrast; while in the deep shadows of his eyes floated some\\nreminiscences that did not seem to give him much joy.  His voice at\\nonce announced that he was a Southerner, and from his fine stature, I\\nthought he must be one of those tall mountaineers from the\\nAlleghanian Ridge in Virginia.  When the revelry of his companions\\nhad mounted to its height, this man slipped away unobserved, and I\\nsaw no more of him till he became my comrade on the sea.  In a few\\nminutes, however, he was missed by his shipmates, and being, it\\nseems, for some reason a huge favourite with them, they raised a cry\\nof \"Bulkington!  Bulkington! where\\'s Bulkington?\" and darted out of\\nthe house in pursuit of him.\\n\\nIt was now about nine o\\'clock, and the room seeming almost\\nsupernaturally quiet after these orgies, I began to congratulate\\nmyself upon a little plan that had occurred to me just previous to\\nthe entrance of the seamen.\\n\\nNo man prefers to sleep two in a bed.  In fact, you would a good deal\\nrather not sleep with your own brother.  I don\\'t know how it is, but\\npeople like to be private when they are sleeping.  And when it comes\\nto sleeping with an unknown stranger, in a strange inn, in a strange\\ntown, and that stranger a harpooneer, then your objections\\nindefinitely multiply.  Nor was there any earthly reason why I as a\\nsailor should sleep two in a bed, more than anybody else; for sailors\\nno more sleep two in a bed at sea, than bachelor Kings do ashore.  To\\nbe sure they all sleep together in one apartment, but you have your\\nown hammock, and cover yourself with your own blanket, and sleep in\\nyour own skin.\\n\\nThe more I pondered over this harpooneer, the more I abominated the\\nthought of sleeping with him.  It was fair to presume that being a\\nharpooneer, his linen or woollen, as the case might be, would not be\\nof the tidiest, certainly none of the finest.  I began to twitch all\\nover.  Besides, it was getting late, and my decent harpooneer ought\\nto be home and going bedwards.  Suppose now, he should tumble in upon\\nme at midnight--how could I tell from what vile hole he had been\\ncoming?\\n\\n\"Landlord!  I\\'ve changed my mind about that harpooneer.--I shan\\'t\\nsleep with him.  I\\'ll try the bench here.\"\\n\\n\"Just as you please; I\\'m sorry I cant spare ye a tablecloth for a\\nmattress, and it\\'s a plaguy rough board here\"--feeling of the knots\\nand notches.  \"But wait a bit, Skrimshander; I\\'ve got a carpenter\\'s\\nplane there in the bar--wait, I say, and I\\'ll make ye snug enough.\"\\nSo saying he procured the plane; and with his old silk handkerchief\\nfirst dusting the bench, vigorously set to planing away at my bed,\\nthe while grinning like an ape.  The shavings flew right and left;\\ntill at last the plane-iron came bump against an indestructible knot.\\nThe landlord was near spraining his wrist, and I told him for\\nheaven\\'s sake to quit--the bed was soft enough to suit me, and I did\\nnot know how all the planing in the world could make eider down of a\\npine plank.  So gathering up the shavings with another grin, and\\nthrowing them into the great stove in the middle of the room, he went\\nabout his business, and left me in a brown study.\\n\\nI now took the measure of the bench, and found that it was a foot too\\nshort; but that could be mended with a chair.  But it was a foot too\\nnarrow, and the other bench in the room was about four inches higher\\nthan the planed one--so there was no yoking them.  I then placed the\\nfirst bench lengthwise along the only clear space against the wall,\\nleaving a little interval between, for my back to settle down in.\\nBut I soon found that there came such a draught of cold air over me\\nfrom under the sill of the window, that this plan would never do at\\nall, especially as another current from the rickety door met the one\\nfrom the window, and both together formed a series of small\\nwhirlwinds in the immediate vicinity of the spot where I had thought\\nto spend the night.\\n\\nThe devil fetch that harpooneer, thought I, but stop, couldn\\'t I\\nsteal a march on him--bolt his door inside, and jump into his bed,\\nnot to be wakened by the most violent knockings?  It seemed no bad\\nidea; but upon second thoughts I dismissed it.  For who could tell\\nbut what the next morning, so soon as I popped out of the room, the\\nharpooneer might be standing in the entry, all ready to knock me\\ndown!\\n\\nStill, looking round me again, and seeing no possible chance of\\nspending a sufferable night unless in some other person\\'s bed, I\\nbegan to think that after all I might be cherishing unwarrantable\\nprejudices against this unknown harpooneer.  Thinks I, I\\'ll wait\\nawhile; he must be dropping in before long.  I\\'ll have a good look at\\nhim then, and perhaps we may become jolly good bedfellows after\\nall--there\\'s no telling.\\n\\nBut though the other boarders kept coming in by ones, twos, and\\nthrees, and going to bed, yet no sign of my harpooneer.\\n\\n\"Landlord! said I, \"what sort of a chap is he--does he always keep\\nsuch late hours?\"  It was now hard upon twelve o\\'clock.\\n\\nThe landlord chuckled again with his lean chuckle, and seemed to be\\nmightily tickled at something beyond my comprehension.  \"No,\" he\\nanswered, \"generally he\\'s an early bird--airley to bed and airley to\\nrise--yes, he\\'s the bird what catches the worm.  But to-night he\\nwent out a peddling, you see, and I don\\'t see what on airth keeps him\\nso late, unless, may be, he can\\'t sell his head.\"\\n\\n\"Can\\'t sell his head?--What sort of a bamboozingly story is this you\\nare telling me?\" getting into a towering rage.  \"Do you pretend to\\nsay, landlord, that this harpooneer is actually engaged this blessed\\nSaturday night, or rather Sunday morning, in peddling his head around\\nthis town?\"\\n\\n\"That\\'s precisely it,\" said the landlord, \"and I told him he couldn\\'t\\nsell it here, the market\\'s overstocked.\"\\n\\n\"With what?\" shouted I.\\n\\n\"With heads to be sure; ain\\'t there too many heads in the world?\"\\n\\n\"I tell you what it is, landlord,\" said I quite calmly, \"you\\'d better\\nstop spinning that yarn to me--I\\'m not green.\"\\n\\n\"May be not,\" taking out a stick and whittling a toothpick, \"but I\\nrayther guess you\\'ll be done BROWN if that ere harpooneer hears you a\\nslanderin\\' his head.\"\\n\\n\"I\\'ll break it for him,\" said I, now flying into a passion again at\\nthis unaccountable farrago of the landlord\\'s.\\n\\n\"It\\'s broke a\\'ready,\" said he.\\n\\n\"Broke,\" said I--\"BROKE, do you mean?\"\\n\\n\"Sartain, and that\\'s the very reason he can\\'t sell it, I guess.\"\\n\\n\"Landlord,\" said I, going up to him as cool as Mt. Hecla in a\\nsnow-storm--\"landlord, stop whittling.  You and I must understand one\\nanother, and that too without delay.  I come to your house and want a\\nbed; you tell me you can only give me half a one; that the other half\\nbelongs to a certain harpooneer.  And about this harpooneer, whom I\\nhave not yet seen, you persist in telling me the most mystifying and\\nexasperating stories tending to beget in me an uncomfortable feeling\\ntowards the man whom you design for my bedfellow--a sort of\\nconnexion, landlord, which is an intimate and confidential one in the\\nhighest degree.  I now demand of you to speak out and tell me who and\\nwhat this harpooneer is, and whether I shall be in all respects safe\\nto spend the night with him.  And in the first place, you will be so\\ngood as to unsay that story about selling his head, which if true I\\ntake to be good evidence that this harpooneer is stark mad, and I\\'ve\\nno idea of sleeping with a madman; and you, sir, YOU I mean,\\nlandlord, YOU, sir, by trying to induce me to do so knowingly, would\\nthereby render yourself liable to a criminal prosecution.\"\\n\\n\"Wall,\" said the landlord, fetching a long breath, \"that\\'s a purty\\nlong sarmon for a chap that rips a little now and then.  But be easy,\\nbe easy, this here harpooneer I have been tellin\\' you of has just\\narrived from the south seas, where he bought up a lot of \\'balmed New\\nZealand heads (great curios, you know), and he\\'s sold all on \\'em but\\none, and that one he\\'s trying to sell to-night, cause to-morrow\\'s\\nSunday, and it would not do to be sellin\\' human heads about the\\nstreets when folks is goin\\' to churches.  He wanted to, last Sunday,\\nbut I stopped him just as he was goin\\' out of the door with four\\nheads strung on a string, for all the airth like a string of inions.\"\\n\\nThis account cleared up the otherwise unaccountable mystery, and\\nshowed that the landlord, after all, had had no idea of fooling\\nme--but at the same time what could I think of a harpooneer who\\nstayed out of a Saturday night clean into the holy Sabbath, engaged\\nin such a cannibal business as selling the heads of dead idolators?\\n\\n\"Depend upon it, landlord, that harpooneer is a dangerous man.\"\\n\\n\"He pays reg\\'lar,\" was the rejoinder.  \"But come, it\\'s getting\\ndreadful late, you had better be turning flukes--it\\'s a nice bed;\\nSal and me slept in that ere bed the night we were spliced.  There\\'s\\nplenty of room for two to kick about in that bed; it\\'s an almighty\\nbig bed that.  Why, afore we give it up, Sal used to put our Sam and\\nlittle Johnny in the foot of it.  But I got a dreaming and sprawling\\nabout one night, and somehow, Sam got pitched on the floor, and came\\nnear breaking his arm.  Arter that, Sal said it wouldn\\'t do.  Come\\nalong here, I\\'ll give ye a glim in a jiffy;\" and so saying he lighted\\na candle and held it towards me, offering to lead the way.  But I\\nstood irresolute; when looking at a clock in the corner, he exclaimed\\n\"I vum it\\'s Sunday--you won\\'t see that harpooneer to-night; he\\'s come\\nto anchor somewhere--come along then; DO come; WON\\'T ye come?\"\\n\\nI considered the matter a moment, and then up stairs we went, and I\\nwas ushered into a small room, cold as a clam, and furnished, sure\\nenough, with a prodigious bed, almost big enough indeed for any four\\nharpooneers to sleep abreast.\\n\\n\"There,\" said the landlord, placing the candle on a crazy old sea\\nchest that did double duty as a wash-stand and centre table; \"there,\\nmake yourself comfortable now, and good night to ye.\"  I turned\\nround from eyeing the bed, but he had disappeared.\\n\\nFolding back the counterpane, I stooped over the bed.  Though none of\\nthe most elegant, it yet stood the scrutiny tolerably well.  I then\\nglanced round the room; and besides the bedstead and centre table,\\ncould see no other furniture belonging to the place, but a rude\\nshelf, the four walls, and a papered fireboard representing a man\\nstriking a whale.  Of things not properly belonging to the room,\\nthere was a hammock lashed up, and thrown upon the floor in one\\ncorner; also a large seaman\\'s bag, containing the harpooneer\\'s\\nwardrobe, no doubt in lieu of a land trunk.  Likewise, there was a\\nparcel of outlandish bone fish hooks on the shelf over the\\nfire-place, and a tall harpoon standing at the head of the bed.\\n\\nBut what is this on the chest?  I took it up, and held it close to\\nthe light, and felt it, and smelt it, and tried every way possible to\\narrive at some satisfactory conclusion concerning it.  I can compare\\nit to nothing but a large door mat, ornamented at the edges with\\nlittle tinkling tags something like the stained porcupine quills\\nround an Indian moccasin.  There was a hole or slit in the middle of\\nthis mat, as you see the same in South American ponchos.  But could\\nit be possible that any sober harpooneer would get into a door mat,\\nand parade the streets of any Christian town in that sort of guise?\\nI put it on, to try it, and it weighed me down like a hamper, being\\nuncommonly shaggy and thick, and I thought a little damp, as though\\nthis mysterious harpooneer had been wearing it of a rainy day.  I\\nwent up in it to a bit of glass stuck against the wall, and I never\\nsaw such a sight in my life.  I tore myself out of it in such a hurry\\nthat I gave myself a kink in the neck.\\n\\nI sat down on the side of the bed, and commenced thinking about this\\nhead-peddling harpooneer, and his door mat.  After thinking some time\\non the bed-side, I got up and took off my monkey jacket, and then\\nstood in the middle of the room thinking.  I then took off my coat,\\nand thought a little more in my shirt sleeves.  But beginning to feel\\nvery cold now, half undressed as I was, and remembering what the\\nlandlord said about the harpooneer\\'s not coming home at all that\\nnight, it being so very late, I made no more ado, but jumped out of\\nmy pantaloons and boots, and then blowing out the light tumbled into\\nbed, and commended myself to the care of heaven.\\n\\nWhether that mattress was stuffed with corn-cobs or broken crockery,\\nthere is no telling, but I rolled about a good deal, and could not\\nsleep for a long time.  At last I slid off into a light doze, and had\\npretty nearly made a good offing towards the land of Nod, when I\\nheard a heavy footfall in the passage, and saw a glimmer of light\\ncome into the room from under the door.\\n\\nLord save me, thinks I, that must be the harpooneer, the infernal\\nhead-peddler.  But I lay perfectly still, and resolved not to say a\\nword till spoken to.  Holding a light in one hand, and that identical\\nNew Zealand head in the other, the stranger entered the room, and\\nwithout looking towards the bed, placed his candle a good way off\\nfrom me on the floor in one corner, and then began working away at\\nthe knotted cords of the large bag I before spoke of as being in the\\nroom.  I was all eagerness to see his face, but he kept it averted\\nfor some time while employed in unlacing the bag\\'s mouth.  This\\naccomplished, however, he turned round--when, good heavens! what a\\nsight!  Such a face!  It was of a dark, purplish, yellow colour, here\\nand there stuck over with large blackish looking squares.  Yes, it\\'s\\njust as I thought, he\\'s a terrible bedfellow; he\\'s been in a fight,\\ngot dreadfully cut, and here he is, just from the surgeon.  But at\\nthat moment he chanced to turn his face so towards the light, that I\\nplainly saw they could not be sticking-plasters at all, those black\\nsquares on his cheeks.  They were stains of some sort or other.  At\\nfirst I knew not what to make of this; but soon an inkling of the\\ntruth occurred to me.  I remembered a story of a white man--a\\nwhaleman too--who, falling among the cannibals, had been tattooed by\\nthem.  I concluded that this harpooneer, in the course of his distant\\nvoyages, must have met with a similar adventure.  And what is it,\\nthought I, after all!  It\\'s only his outside; a man can be honest in\\nany sort of skin.  But then, what to make of his unearthly\\ncomplexion, that part of it, I mean, lying round about, and\\ncompletely independent of the squares of tattooing.  To be sure, it\\nmight be nothing but a good coat of tropical tanning; but I never\\nheard of a hot sun\\'s tanning a white man into a purplish yellow one.\\nHowever, I had never been in the South Seas; and perhaps the sun\\nthere produced these extraordinary effects upon the skin.  Now, while\\nall these ideas were passing through me like lightning, this\\nharpooneer never noticed me at all.  But, after some difficulty\\nhaving opened his bag, he commenced fumbling in it, and presently\\npulled out a sort of tomahawk, and a seal-skin wallet with the hair\\non.  Placing these on the old chest in the middle of the room, he\\nthen took the New Zealand head--a ghastly thing enough--and crammed\\nit down into the bag.  He now took off his hat--a new beaver\\nhat--when I came nigh singing out with fresh surprise.  There was no\\nhair on his head--none to speak of at least--nothing but a small\\nscalp-knot twisted up on his forehead.  His bald purplish head now\\nlooked for all the world like a mildewed skull.  Had not the stranger\\nstood between me and the door, I would have bolted out of it quicker\\nthan ever I bolted a dinner.\\n\\nEven as it was, I thought something of slipping out of the window,\\nbut it was the second floor back.  I am no coward, but what to make\\nof this head-peddling purple rascal altogether passed my\\ncomprehension.  Ignorance is the parent of fear, and being completely\\nnonplussed and confounded about the stranger, I confess I was now as\\nmuch afraid of him as if it was the devil himself who had thus broken\\ninto my room at the dead of night.  In fact, I was so afraid of him\\nthat I was not game enough just then to address him, and demand a\\nsatisfactory answer concerning what seemed inexplicable in him.\\n\\nMeanwhile, he continued the business of undressing, and at last\\nshowed his chest and arms.  As I live, these covered parts of him\\nwere checkered with the same squares as his face; his back, too, was\\nall over the same dark squares; he seemed to have been in a Thirty\\nYears\\' War, and just escaped from it with a sticking-plaster shirt.\\nStill more, his very legs were marked, as if a parcel of dark green\\nfrogs were running up the trunks of young palms.  It was now quite\\nplain that he must be some abominable savage or other shipped aboard\\nof a whaleman in the South Seas, and so landed in this Christian\\ncountry.  I quaked to think of it.  A peddler of heads too--perhaps\\nthe heads of his own brothers.  He might take a fancy to\\nmine--heavens! look at that tomahawk!\\n\\nBut there was no time for shuddering, for now the savage went about\\nsomething that completely fascinated my attention, and convinced me\\nthat he must indeed be a heathen.  Going to his heavy grego, or\\nwrapall, or dreadnaught, which he had previously hung on a chair, he\\nfumbled in the pockets, and produced at length a curious little\\ndeformed image with a hunch on its back, and exactly the colour of a\\nthree days\\' old Congo baby.  Remembering the embalmed head, at first\\nI almost thought that this black manikin was a real baby preserved\\nin some similar manner.  But seeing that it was not at all limber,\\nand that it glistened a good deal like polished ebony, I concluded\\nthat it must be nothing but a wooden idol, which indeed it proved to\\nbe.  For now the savage goes up to the empty fire-place, and removing\\nthe papered fire-board, sets up this little hunch-backed image, like\\na tenpin, between the andirons.  The chimney jambs and all the bricks\\ninside were very sooty, so that I thought this fire-place made a very\\nappropriate little shrine or chapel for his Congo idol.\\n\\nI now screwed my eyes hard towards the half hidden image, feeling but\\nill at ease meantime--to see what was next to follow.  First he takes\\nabout a double handful of shavings out of his grego pocket, and\\nplaces them carefully before the idol; then laying a bit of ship\\nbiscuit on top and applying the flame from the lamp, he kindled the\\nshavings into a sacrificial blaze.  Presently, after many hasty\\nsnatches into the fire, and still hastier withdrawals of his fingers\\n(whereby he seemed to be scorching them badly), he at last succeeded\\nin drawing out the biscuit; then blowing off the heat and ashes a\\nlittle, he made a polite offer of it to the little negro.  But the\\nlittle devil did not seem to fancy such dry sort of fare at all; he\\nnever moved his lips.  All these strange antics were accompanied by\\nstill stranger guttural noises from the devotee, who seemed to be\\npraying in a sing-song or else singing some pagan psalmody or other,\\nduring which his face twitched about in the most unnatural manner.\\nAt last extinguishing the fire, he took the idol up very\\nunceremoniously, and bagged it again in his grego pocket as\\ncarelessly as if he were a sportsman bagging a dead woodcock.\\n\\nAll these queer proceedings increased my uncomfortableness, and\\nseeing him now exhibiting strong symptoms of concluding his business\\noperations, and jumping into bed with me, I thought it was high time,\\nnow or never, before the light was put out, to break the spell in\\nwhich I had so long been bound.\\n\\nBut the interval I spent in deliberating what to say, was a fatal\\none.  Taking up his tomahawk from the table, he examined the head of\\nit for an instant, and then holding it to the light, with his mouth\\nat the handle, he puffed out great clouds of tobacco smoke.  The next\\nmoment the light was extinguished, and this wild cannibal, tomahawk\\nbetween his teeth, sprang into bed with me.  I sang out, I could not\\nhelp it now; and giving a sudden grunt of astonishment he began\\nfeeling me.\\n\\nStammering out something, I knew not what, I rolled away from him\\nagainst the wall, and then conjured him, whoever or whatever he might\\nbe, to keep quiet, and let me get up and light the lamp again.  But\\nhis guttural responses satisfied me at once that he but ill\\ncomprehended my meaning.\\n\\n\"Who-e debel you?\"--he at last said--\"you no speak-e, dam-me, I\\nkill-e.\"  And so saying the lighted tomahawk began flourishing about\\nme in the dark.\\n\\n\"Landlord, for God\\'s sake, Peter Coffin!\" shouted I.  \"Landlord!\\nWatch!  Coffin!  Angels! save me!\"\\n\\n\"Speak-e! tell-ee me who-ee be, or dam-me, I kill-e!\" again growled\\nthe cannibal, while his horrid flourishings of the tomahawk scattered\\nthe hot tobacco ashes about me till I thought my linen would get on\\nfire.  But thank heaven, at that moment the landlord came into the\\nroom light in hand, and leaping from the bed I ran up to him.\\n\\n\"Don\\'t be afraid now,\" said he, grinning again, \"Queequeg here\\nwouldn\\'t harm a hair of your head.\"\\n\\n\"Stop your grinning,\" shouted I, \"and why didn\\'t you tell me that\\nthat infernal harpooneer was a cannibal?\"\\n\\n\"I thought ye know\\'d it;--didn\\'t I tell ye, he was a peddlin\\' heads\\naround town?--but turn flukes again and go to sleep.  Queequeg, look\\nhere--you sabbee me, I sabbee--you this man sleepe you--you sabbee?\"\\n\\n\"Me sabbee plenty\"--grunted Queequeg, puffing away at his pipe and\\nsitting up in bed.\\n\\n\"You gettee in,\" he added, motioning to me with his tomahawk, and\\nthrowing the clothes to one side.  He really did this in not only a\\ncivil but a really kind and charitable way.  I stood looking at him a\\nmoment.  For all his tattooings he was on the whole a clean, comely\\nlooking cannibal.  What\\'s all this fuss I have been making about,\\nthought I to myself--the man\\'s a human being just as I am: he has\\njust as much reason to fear me, as I have to be afraid of him.\\nBetter sleep with a sober cannibal than a drunken Christian.\\n\\n\"Landlord,\" said I, \"tell him to stash his tomahawk there, or pipe,\\nor whatever you call it; tell him to stop smoking, in short, and I\\nwill turn in with him.  But I don\\'t fancy having a man smoking in bed\\nwith me.  It\\'s dangerous.  Besides, I ain\\'t insured.\"\\n\\nThis being told to Queequeg, he at once complied, and again politely\\nmotioned me to get into bed--rolling over to one side as much as to\\nsay--I won\\'t touch a leg of ye.\"\\n\\n\"Good night, landlord,\" said I, \"you may go.\"\\n\\nI turned in, and never slept better in my life.\\n\\nUpon waking next morning about daylight, I found Queequeg\\'s arm\\nthrown over me in the most loving and affectionate manner.  You had\\nalmost thought I had been his wife.  The counterpane was of\\npatchwork, full of odd little parti-coloured squares and triangles;\\nand this arm of his tattooed all over with an interminable Cretan\\nlabyrinth of a figure, no two parts of which were of one precise\\nshade--owing I suppose to his keeping his arm at sea unmethodically\\nin sun and shade, his shirt sleeves irregularly rolled up at various\\ntimes--this same arm of his, I say, looked for all the world like a\\nstrip of that same patchwork quilt.  Indeed, partly lying on it as\\nthe arm did when I first awoke, I could hardly tell it from the\\nquilt, they so blended their hues together; and it was only by the\\nsense of weight and pressure that I could tell that Queequeg was\\nhugging me.\\n\\nMy sensations were strange.  Let me try to explain them.  When I was\\na child, I well remember a somewhat similar circumstance that befell\\nme; whether it was a reality or a dream, I never could entirely\\nsettle.  The circumstance was this.  I had been cutting up some caper\\nor other--I think it was trying to crawl up the chimney, as I had\\nseen a little sweep do a few days previous; and my stepmother who,\\nsomehow or other, was all the time whipping me, or sending me to bed\\nsupperless,--my mother dragged me by the legs out of the chimney and\\npacked me off to bed, though it was only two o\\'clock in the afternoon\\nof the 21st June, the longest day in the year in our hemisphere.  I\\nfelt dreadfully.  But there was no help for it, so up stairs I went\\nto my little room in the third floor, undressed myself as slowly as\\npossible so as to kill time, and with a bitter sigh got between the\\nsheets.\\n\\nI lay there dismally calculating that sixteen entire hours must\\nelapse before I could hope for a resurrection.  Sixteen hours in bed!\\nthe small of my back ached to think of it.  And it was so light too;\\nthe sun shining in at the window, and a great rattling of coaches in\\nthe streets, and the sound of gay voices all over the house.  I felt\\nworse and worse--at last I got up, dressed, and softly going down in\\nmy stockinged feet, sought out my stepmother, and suddenly threw\\nmyself at her feet, beseeching her as a particular favour to give me a\\ngood slippering for my misbehaviour; anything indeed but condemning\\nme to lie abed such an unendurable length of time.  But she was the\\nbest and most conscientious of stepmothers, and back I had to go to\\nmy room.  For several hours I lay there broad awake, feeling a great\\ndeal worse than I have ever done since, even from the greatest\\nsubsequent misfortunes.  At last I must have fallen into a troubled\\nnightmare of a doze; and slowly waking from it--half steeped in\\ndreams--I opened my eyes, and the before sun-lit room was now wrapped\\nin outer darkness.  Instantly I felt a shock running through all my\\nframe; nothing was to be seen, and nothing was to be heard; but a\\nsupernatural hand seemed placed in mine.  My arm hung over the\\ncounterpane, and the nameless, unimaginable, silent form or phantom,\\nto which the hand belonged, seemed closely seated by my bed-side.\\nFor what seemed ages piled on ages, I lay there, frozen with the most\\nawful fears, not daring to drag away my hand; yet ever thinking that\\nif I could but stir it one single inch, the horrid spell would be\\nbroken.  I knew not how this consciousness at last glided away from\\nme; but waking in the morning, I shudderingly remembered it all, and\\nfor days and weeks and months afterwards I lost myself in confounding\\nattempts to explain the mystery.  Nay, to this very hour, I often\\npuzzle myself with it.\\n\\nNow, take away the awful fear, and my sensations at feeling the\\nsupernatural hand in mine were very similar, in their strangeness,\\nto those which I experienced on waking up and seeing Queequeg\\'s pagan\\narm thrown round me.  But at length all the past night\\'s events\\nsoberly recurred, one by one, in fixed reality, and then I lay only\\nalive to the comical predicament.  For though I tried to move his\\narm--unlock his bridegroom clasp--yet, sleeping as he was, he still\\nhugged me tightly, as though naught but death should part us twain.\\nI now strove to rouse him--\"Queequeg!\"--but his only answer was a\\nsnore.  I then rolled over, my neck feeling as if it were in a\\nhorse-collar; and suddenly felt a slight scratch.  Throwing aside the\\ncounterpane, there lay the tomahawk sleeping by the savage\\'s side, as\\nif it were a hatchet-faced baby.  A pretty pickle, truly, thought I;\\nabed here in a strange house in the broad day, with a cannibal and a\\ntomahawk!  \"Queequeg!--in the name of goodness, Queequeg, wake!\"  At\\nlength, by dint of much wriggling, and loud and incessant\\nexpostulations upon the unbecomingness of his hugging a fellow male\\nin that matrimonial sort of style, I succeeded in extracting a grunt;\\nand presently, he drew back his arm, shook himself all over like a\\nNewfoundland dog just from the water, and sat up in bed, stiff as a\\npike-staff, looking at me, and rubbing his eyes as if he did not\\naltogether remember how I came to be there, though a dim\\nconsciousness of knowing something about me seemed slowly dawning\\nover him.  Meanwhile, I lay quietly eyeing him, having no serious\\nmisgivings now, and bent upon narrowly observing so curious a\\ncreature.  When, at last, his mind seemed made up touching the\\ncharacter of his bedfellow, and he became, as it were, reconciled to\\nthe fact; he jumped out upon the floor, and by certain signs and\\nsounds gave me to understand that, if it pleased me, he would dress\\nfirst and then leave me to dress afterwards, leaving the whole\\napartment to myself.  Thinks I, Queequeg, under the circumstances,\\nthis is a very civilized overture; but, the truth is, these savages\\nhave an innate sense of delicacy, say what you will; it is marvellous\\nhow essentially polite they are.  I pay this particular compliment to\\nQueequeg, because he treated me with so much civility and\\nconsideration, while I was guilty of great rudeness; staring at him\\nfrom the bed, and watching all his toilette motions; for the time my\\ncuriosity getting the better of my breeding.  Nevertheless, a man\\nlike Queequeg you don\\'t see every day, he and his ways were well\\nworth unusual regarding.\\n\\nHe commenced dressing at top by donning his beaver hat, a very tall\\none, by the by, and then--still minus his trowsers--he hunted up his\\nboots.  What under the heavens he did it for, I cannot tell, but his\\nnext movement was to crush himself--boots in hand, and hat on--under\\nthe bed; when, from sundry violent gaspings and strainings, I\\ninferred he was hard at work booting himself; though by no law of\\npropriety that I ever heard of, is any man required to be private\\nwhen putting on his boots.  But Queequeg, do you see, was a creature\\nin the transition stage--neither caterpillar nor butterfly.  He was\\njust enough civilized to show off his outlandishness in the strangest\\npossible manners.  His education was not yet completed.  He was an\\nundergraduate.  If he had not been a small degree civilized, he very\\nprobably would not have troubled himself with boots at all; but then,\\nif he had not been still a savage, he never would have dreamt of\\ngetting under the bed to put them on.  At last, he emerged with his\\nhat very much dented and crushed down over his eyes, and began\\ncreaking and limping about the room, as if, not being much accustomed\\nto boots, his pair of damp, wrinkled cowhide ones--probably not made\\nto order either--rather pinched and tormented him at the first go off\\nof a bitter cold morning.\\n\\nSeeing, now, that there were no curtains to the window, and that the\\nstreet being very narrow, the house opposite commanded a plain view\\ninto the room, and observing more and more the indecorous figure that\\nQueequeg made, staving about with little else but his hat and boots\\non; I begged him as well as I could, to accelerate his toilet\\nsomewhat, and particularly to get into his pantaloons as soon as\\npossible.  He complied, and then proceeded to wash himself.  At that\\ntime in the morning any Christian would have washed his face; but\\nQueequeg, to my amazement, contented himself with restricting his\\nablutions to his chest, arms, and hands.  He then donned his\\nwaistcoat, and taking up a piece of hard soap on the wash-stand\\ncentre table, dipped it into water and commenced lathering his face.\\nI was watching to see where he kept his razor, when lo and behold, he\\ntakes the harpoon from the bed corner, slips out the long wooden\\nstock, unsheathes the head, whets it a little on his boot, and\\nstriding up to the bit of mirror against the wall, begins a vigorous\\nscraping, or rather harpooning of his cheeks.  Thinks I, Queequeg,\\nthis is using Rogers\\'s best cutlery with a vengeance.  Afterwards I\\nwondered the less at this operation when I came to know of what fine\\nsteel the head of a harpoon is made, and how exceedingly sharp the\\nlong straight edges are always kept.\\n\\nThe rest of his toilet was soon achieved, and he proudly marched out\\nof the room, wrapped up in his great pilot monkey jacket, and\\nsporting his harpoon like a marshal\\'s baton.'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#read_file('moby_dick_four_chapters.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import spacy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "nlp = spacy.load('en', disable = ['parser', 'tagger', 'ner'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "nlp.max_length = 1198623"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def separate_punc(doc_text):\n",
    "    return [token.text.lower() for token in nlp(doc_text) if token.text not in '\\n\\n \\n\\n\\n!\"-#$%&()--.*+,-/:;<=>?@[\\\\]^_`{|}~\\t\\n ']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "d = read_file('moby_dick_four_chapters.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "tokens = separate_punc(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['call',\n",
       " 'me',\n",
       " 'ishmael',\n",
       " 'some',\n",
       " 'years',\n",
       " 'ago',\n",
       " 'never',\n",
       " 'mind',\n",
       " 'how',\n",
       " 'long',\n",
       " 'precisely',\n",
       " 'having',\n",
       " 'little',\n",
       " 'or',\n",
       " 'no',\n",
       " 'money',\n",
       " 'in',\n",
       " 'my',\n",
       " 'purse',\n",
       " 'and',\n",
       " 'nothing',\n",
       " 'particular',\n",
       " 'to',\n",
       " 'interest',\n",
       " 'me',\n",
       " 'on',\n",
       " 'shore',\n",
       " 'i',\n",
       " 'thought',\n",
       " 'i',\n",
       " 'would',\n",
       " 'sail',\n",
       " 'about',\n",
       " 'a',\n",
       " 'little',\n",
       " 'and',\n",
       " 'see',\n",
       " 'the',\n",
       " 'watery',\n",
       " 'part',\n",
       " 'of',\n",
       " 'the',\n",
       " 'world',\n",
       " 'it',\n",
       " 'is',\n",
       " 'a',\n",
       " 'way',\n",
       " 'i',\n",
       " 'have',\n",
       " 'of',\n",
       " 'driving',\n",
       " 'off',\n",
       " 'the',\n",
       " 'spleen',\n",
       " 'and',\n",
       " 'regulating',\n",
       " 'the',\n",
       " 'circulation',\n",
       " 'whenever',\n",
       " 'i',\n",
       " 'find',\n",
       " 'myself',\n",
       " 'growing',\n",
       " 'grim',\n",
       " 'about',\n",
       " 'the',\n",
       " 'mouth',\n",
       " 'whenever',\n",
       " 'it',\n",
       " 'is',\n",
       " 'a',\n",
       " 'damp',\n",
       " 'drizzly',\n",
       " 'november',\n",
       " 'in',\n",
       " 'my',\n",
       " 'soul',\n",
       " 'whenever',\n",
       " 'i',\n",
       " 'find',\n",
       " 'myself',\n",
       " 'involuntarily',\n",
       " 'pausing',\n",
       " 'before',\n",
       " 'coffin',\n",
       " 'warehouses',\n",
       " 'and',\n",
       " 'bringing',\n",
       " 'up',\n",
       " 'the',\n",
       " 'rear',\n",
       " 'of',\n",
       " 'every',\n",
       " 'funeral',\n",
       " 'i',\n",
       " 'meet',\n",
       " 'and',\n",
       " 'especially',\n",
       " 'whenever',\n",
       " 'my',\n",
       " 'hypos',\n",
       " 'get',\n",
       " 'such',\n",
       " 'an',\n",
       " 'upper',\n",
       " 'hand',\n",
       " 'of',\n",
       " 'me',\n",
       " 'that',\n",
       " 'it',\n",
       " 'requires',\n",
       " 'a',\n",
       " 'strong',\n",
       " 'moral',\n",
       " 'principle',\n",
       " 'to',\n",
       " 'prevent',\n",
       " 'me',\n",
       " 'from',\n",
       " 'deliberately',\n",
       " 'stepping',\n",
       " 'into',\n",
       " 'the',\n",
       " 'street',\n",
       " 'and',\n",
       " 'methodically',\n",
       " 'knocking',\n",
       " 'people',\n",
       " \"'s\",\n",
       " 'hats',\n",
       " 'off',\n",
       " 'then',\n",
       " 'i',\n",
       " 'account',\n",
       " 'it',\n",
       " 'high',\n",
       " 'time',\n",
       " 'to',\n",
       " 'get',\n",
       " 'to',\n",
       " 'sea',\n",
       " 'as',\n",
       " 'soon',\n",
       " 'as',\n",
       " 'i',\n",
       " 'can',\n",
       " 'this',\n",
       " 'is',\n",
       " 'my',\n",
       " 'substitute',\n",
       " 'for',\n",
       " 'pistol',\n",
       " 'and',\n",
       " 'ball',\n",
       " 'with',\n",
       " 'a',\n",
       " 'philosophical',\n",
       " 'flourish',\n",
       " 'cato',\n",
       " 'throws',\n",
       " 'himself',\n",
       " 'upon',\n",
       " 'his',\n",
       " 'sword',\n",
       " 'i',\n",
       " 'quietly',\n",
       " 'take',\n",
       " 'to',\n",
       " 'the',\n",
       " 'ship',\n",
       " 'there',\n",
       " 'is',\n",
       " 'nothing',\n",
       " 'surprising',\n",
       " 'in',\n",
       " 'this',\n",
       " 'if',\n",
       " 'they',\n",
       " 'but',\n",
       " 'knew',\n",
       " 'it',\n",
       " 'almost',\n",
       " 'all',\n",
       " 'men',\n",
       " 'in',\n",
       " 'their',\n",
       " 'degree',\n",
       " 'some',\n",
       " 'time',\n",
       " 'or',\n",
       " 'other',\n",
       " 'cherish',\n",
       " 'very',\n",
       " 'nearly',\n",
       " 'the',\n",
       " 'same',\n",
       " 'feelings',\n",
       " 'towards',\n",
       " 'the',\n",
       " 'ocean',\n",
       " 'with',\n",
       " 'me',\n",
       " 'there',\n",
       " 'now',\n",
       " 'is',\n",
       " 'your',\n",
       " 'insular',\n",
       " 'city',\n",
       " 'of',\n",
       " 'the',\n",
       " 'manhattoes',\n",
       " 'belted',\n",
       " 'round',\n",
       " 'by',\n",
       " 'wharves',\n",
       " 'as',\n",
       " 'indian',\n",
       " 'isles',\n",
       " 'by',\n",
       " 'coral',\n",
       " 'reefs',\n",
       " 'commerce',\n",
       " 'surrounds',\n",
       " 'it',\n",
       " 'with',\n",
       " 'her',\n",
       " 'surf',\n",
       " 'right',\n",
       " 'and',\n",
       " 'left',\n",
       " 'the',\n",
       " 'streets',\n",
       " 'take',\n",
       " 'you',\n",
       " 'waterward',\n",
       " 'its',\n",
       " 'extreme',\n",
       " 'downtown',\n",
       " 'is',\n",
       " 'the',\n",
       " 'battery',\n",
       " 'where',\n",
       " 'that',\n",
       " 'noble',\n",
       " 'mole',\n",
       " 'is',\n",
       " 'washed',\n",
       " 'by',\n",
       " 'waves',\n",
       " 'and',\n",
       " 'cooled',\n",
       " 'by',\n",
       " 'breezes',\n",
       " 'which',\n",
       " 'a',\n",
       " 'few',\n",
       " 'hours',\n",
       " 'previous',\n",
       " 'were',\n",
       " 'out',\n",
       " 'of',\n",
       " 'sight',\n",
       " 'of',\n",
       " 'land',\n",
       " 'look',\n",
       " 'at',\n",
       " 'the',\n",
       " 'crowds',\n",
       " 'of',\n",
       " 'water',\n",
       " 'gazers',\n",
       " 'there',\n",
       " 'circumambulate',\n",
       " 'the',\n",
       " 'city',\n",
       " 'of',\n",
       " 'a',\n",
       " 'dreamy',\n",
       " 'sabbath',\n",
       " 'afternoon',\n",
       " 'go',\n",
       " 'from',\n",
       " 'corlears',\n",
       " 'hook',\n",
       " 'to',\n",
       " 'coenties',\n",
       " 'slip',\n",
       " 'and',\n",
       " 'from',\n",
       " 'thence',\n",
       " 'by',\n",
       " 'whitehall',\n",
       " 'northward',\n",
       " 'what',\n",
       " 'do',\n",
       " 'you',\n",
       " 'see?--posted',\n",
       " 'like',\n",
       " 'silent',\n",
       " 'sentinels',\n",
       " 'all',\n",
       " 'around',\n",
       " 'the',\n",
       " 'town',\n",
       " 'stand',\n",
       " 'thousands',\n",
       " 'upon',\n",
       " 'thousands',\n",
       " 'of',\n",
       " 'mortal',\n",
       " 'men',\n",
       " 'fixed',\n",
       " 'in',\n",
       " 'ocean',\n",
       " 'reveries',\n",
       " 'some',\n",
       " 'leaning',\n",
       " 'against',\n",
       " 'the',\n",
       " 'spiles',\n",
       " 'some',\n",
       " 'seated',\n",
       " 'upon',\n",
       " 'the',\n",
       " 'pier',\n",
       " 'heads',\n",
       " 'some',\n",
       " 'looking',\n",
       " 'over',\n",
       " 'the',\n",
       " 'bulwarks',\n",
       " 'of',\n",
       " 'ships',\n",
       " 'from',\n",
       " 'china',\n",
       " 'some',\n",
       " 'high',\n",
       " 'aloft',\n",
       " 'in',\n",
       " 'the',\n",
       " 'rigging',\n",
       " 'as',\n",
       " 'if',\n",
       " 'striving',\n",
       " 'to',\n",
       " 'get',\n",
       " 'a',\n",
       " 'still',\n",
       " 'better',\n",
       " 'seaward',\n",
       " 'peep',\n",
       " 'but',\n",
       " 'these',\n",
       " 'are',\n",
       " 'all',\n",
       " 'landsmen',\n",
       " 'of',\n",
       " 'week',\n",
       " 'days',\n",
       " 'pent',\n",
       " 'up',\n",
       " 'in',\n",
       " 'lath',\n",
       " 'and',\n",
       " 'plaster',\n",
       " 'tied',\n",
       " 'to',\n",
       " 'counters',\n",
       " 'nailed',\n",
       " 'to',\n",
       " 'benches',\n",
       " 'clinched',\n",
       " 'to',\n",
       " 'desks',\n",
       " 'how',\n",
       " 'then',\n",
       " 'is',\n",
       " 'this',\n",
       " 'are',\n",
       " 'the',\n",
       " 'green',\n",
       " 'fields',\n",
       " 'gone',\n",
       " 'what',\n",
       " 'do',\n",
       " 'they',\n",
       " 'here',\n",
       " 'but',\n",
       " 'look',\n",
       " 'here',\n",
       " 'come',\n",
       " 'more',\n",
       " 'crowds',\n",
       " 'pacing',\n",
       " 'straight',\n",
       " 'for',\n",
       " 'the',\n",
       " 'water',\n",
       " 'and',\n",
       " 'seemingly',\n",
       " 'bound',\n",
       " 'for',\n",
       " 'a',\n",
       " 'dive',\n",
       " 'strange',\n",
       " 'nothing',\n",
       " 'will',\n",
       " 'content',\n",
       " 'them',\n",
       " 'but',\n",
       " 'the',\n",
       " 'extremest',\n",
       " 'limit',\n",
       " 'of',\n",
       " 'the',\n",
       " 'land',\n",
       " 'loitering',\n",
       " 'under',\n",
       " 'the',\n",
       " 'shady',\n",
       " 'lee',\n",
       " 'of',\n",
       " 'yonder',\n",
       " 'warehouses',\n",
       " 'will',\n",
       " 'not',\n",
       " 'suffice',\n",
       " 'no',\n",
       " 'they',\n",
       " 'must',\n",
       " 'get',\n",
       " 'just',\n",
       " 'as',\n",
       " 'nigh',\n",
       " 'the',\n",
       " 'water',\n",
       " 'as',\n",
       " 'they',\n",
       " 'possibly',\n",
       " 'can',\n",
       " 'without',\n",
       " 'falling',\n",
       " 'in',\n",
       " 'and',\n",
       " 'there',\n",
       " 'they',\n",
       " 'stand',\n",
       " 'miles',\n",
       " 'of',\n",
       " 'them',\n",
       " 'leagues',\n",
       " 'inlanders',\n",
       " 'all',\n",
       " 'they',\n",
       " 'come',\n",
       " 'from',\n",
       " 'lanes',\n",
       " 'and',\n",
       " 'alleys',\n",
       " 'streets',\n",
       " 'and',\n",
       " 'avenues',\n",
       " 'north',\n",
       " 'east',\n",
       " 'south',\n",
       " 'and',\n",
       " 'west',\n",
       " 'yet',\n",
       " 'here',\n",
       " 'they',\n",
       " 'all',\n",
       " 'unite',\n",
       " 'tell',\n",
       " 'me',\n",
       " 'does',\n",
       " 'the',\n",
       " 'magnetic',\n",
       " 'virtue',\n",
       " 'of',\n",
       " 'the',\n",
       " 'needles',\n",
       " 'of',\n",
       " 'the',\n",
       " 'compasses',\n",
       " 'of',\n",
       " 'all',\n",
       " 'those',\n",
       " 'ships',\n",
       " 'attract',\n",
       " 'them',\n",
       " 'thither',\n",
       " 'once',\n",
       " 'more',\n",
       " 'say',\n",
       " 'you',\n",
       " 'are',\n",
       " 'in',\n",
       " 'the',\n",
       " 'country',\n",
       " 'in',\n",
       " 'some',\n",
       " 'high',\n",
       " 'land',\n",
       " 'of',\n",
       " 'lakes',\n",
       " 'take',\n",
       " 'almost',\n",
       " 'any',\n",
       " 'path',\n",
       " 'you',\n",
       " 'please',\n",
       " 'and',\n",
       " 'ten',\n",
       " 'to',\n",
       " 'one',\n",
       " 'it',\n",
       " 'carries',\n",
       " 'you',\n",
       " 'down',\n",
       " 'in',\n",
       " 'a',\n",
       " 'dale',\n",
       " 'and',\n",
       " 'leaves',\n",
       " 'you',\n",
       " 'there',\n",
       " 'by',\n",
       " 'a',\n",
       " 'pool',\n",
       " 'in',\n",
       " 'the',\n",
       " 'stream',\n",
       " 'there',\n",
       " 'is',\n",
       " 'magic',\n",
       " 'in',\n",
       " 'it',\n",
       " 'let',\n",
       " 'the',\n",
       " 'most',\n",
       " 'absent',\n",
       " 'minded',\n",
       " 'of',\n",
       " 'men',\n",
       " 'be',\n",
       " 'plunged',\n",
       " 'in',\n",
       " 'his',\n",
       " 'deepest',\n",
       " 'reveries',\n",
       " 'stand',\n",
       " 'that',\n",
       " 'man',\n",
       " 'on',\n",
       " 'his',\n",
       " 'legs',\n",
       " 'set',\n",
       " 'his',\n",
       " 'feet',\n",
       " 'a',\n",
       " 'going',\n",
       " 'and',\n",
       " 'he',\n",
       " 'will',\n",
       " 'infallibly',\n",
       " 'lead',\n",
       " 'you',\n",
       " 'to',\n",
       " 'water',\n",
       " 'if',\n",
       " 'water',\n",
       " 'there',\n",
       " 'be',\n",
       " 'in',\n",
       " 'all',\n",
       " 'that',\n",
       " 'region',\n",
       " 'should',\n",
       " 'you',\n",
       " 'ever',\n",
       " 'be',\n",
       " 'athirst',\n",
       " 'in',\n",
       " 'the',\n",
       " 'great',\n",
       " 'american',\n",
       " 'desert',\n",
       " 'try',\n",
       " 'this',\n",
       " 'experiment',\n",
       " 'if',\n",
       " 'your',\n",
       " 'caravan',\n",
       " 'happen',\n",
       " 'to',\n",
       " 'be',\n",
       " 'supplied',\n",
       " 'with',\n",
       " 'a',\n",
       " 'metaphysical',\n",
       " 'professor',\n",
       " 'yes',\n",
       " 'as',\n",
       " 'every',\n",
       " 'one',\n",
       " 'knows',\n",
       " 'meditation',\n",
       " 'and',\n",
       " 'water',\n",
       " 'are',\n",
       " 'wedded',\n",
       " 'for',\n",
       " 'ever',\n",
       " 'but',\n",
       " 'here',\n",
       " 'is',\n",
       " 'an',\n",
       " 'artist',\n",
       " 'he',\n",
       " 'desires',\n",
       " 'to',\n",
       " 'paint',\n",
       " 'you',\n",
       " 'the',\n",
       " 'dreamiest',\n",
       " 'shadiest',\n",
       " 'quietest',\n",
       " 'most',\n",
       " 'enchanting',\n",
       " 'bit',\n",
       " 'of',\n",
       " 'romantic',\n",
       " 'landscape',\n",
       " 'in',\n",
       " 'all',\n",
       " 'the',\n",
       " 'valley',\n",
       " 'of',\n",
       " 'the',\n",
       " 'saco',\n",
       " 'what',\n",
       " 'is',\n",
       " 'the',\n",
       " 'chief',\n",
       " 'element',\n",
       " 'he',\n",
       " 'employs',\n",
       " 'there',\n",
       " 'stand',\n",
       " 'his',\n",
       " 'trees',\n",
       " 'each',\n",
       " 'with',\n",
       " 'a',\n",
       " 'hollow',\n",
       " 'trunk',\n",
       " 'as',\n",
       " 'if',\n",
       " 'a',\n",
       " 'hermit',\n",
       " 'and',\n",
       " 'a',\n",
       " 'crucifix',\n",
       " 'were',\n",
       " 'within',\n",
       " 'and',\n",
       " 'here',\n",
       " 'sleeps',\n",
       " 'his',\n",
       " 'meadow',\n",
       " 'and',\n",
       " 'there',\n",
       " 'sleep',\n",
       " 'his',\n",
       " 'cattle',\n",
       " 'and',\n",
       " 'up',\n",
       " 'from',\n",
       " 'yonder',\n",
       " 'cottage',\n",
       " 'goes',\n",
       " 'a',\n",
       " 'sleepy',\n",
       " 'smoke',\n",
       " 'deep',\n",
       " 'into',\n",
       " 'distant',\n",
       " 'woodlands',\n",
       " 'winds',\n",
       " 'a',\n",
       " 'mazy',\n",
       " 'way',\n",
       " 'reaching',\n",
       " 'to',\n",
       " 'overlapping',\n",
       " 'spurs',\n",
       " 'of',\n",
       " 'mountains',\n",
       " 'bathed',\n",
       " 'in',\n",
       " 'their',\n",
       " 'hill',\n",
       " 'side',\n",
       " 'blue',\n",
       " 'but',\n",
       " 'though',\n",
       " 'the',\n",
       " 'picture',\n",
       " 'lies',\n",
       " 'thus',\n",
       " 'tranced',\n",
       " 'and',\n",
       " 'though',\n",
       " 'this',\n",
       " 'pine',\n",
       " 'tree',\n",
       " 'shakes',\n",
       " 'down',\n",
       " 'its',\n",
       " 'sighs',\n",
       " 'like',\n",
       " 'leaves',\n",
       " 'upon',\n",
       " 'this',\n",
       " 'shepherd',\n",
       " \"'s\",\n",
       " 'head',\n",
       " 'yet',\n",
       " 'all',\n",
       " 'were',\n",
       " 'vain',\n",
       " 'unless',\n",
       " 'the',\n",
       " 'shepherd',\n",
       " \"'s\",\n",
       " 'eye',\n",
       " 'were',\n",
       " 'fixed',\n",
       " 'upon',\n",
       " 'the',\n",
       " 'magic',\n",
       " 'stream',\n",
       " 'before',\n",
       " 'him',\n",
       " 'go',\n",
       " 'visit',\n",
       " 'the',\n",
       " 'prairies',\n",
       " 'in',\n",
       " 'june',\n",
       " 'when',\n",
       " 'for',\n",
       " 'scores',\n",
       " 'on',\n",
       " 'scores',\n",
       " 'of',\n",
       " 'miles',\n",
       " 'you',\n",
       " 'wade',\n",
       " 'knee',\n",
       " 'deep',\n",
       " 'among',\n",
       " 'tiger',\n",
       " 'lilies',\n",
       " 'what',\n",
       " 'is',\n",
       " 'the',\n",
       " 'one',\n",
       " 'charm',\n",
       " 'wanting?--water',\n",
       " 'there',\n",
       " 'is',\n",
       " 'not',\n",
       " 'a',\n",
       " 'drop',\n",
       " 'of',\n",
       " 'water',\n",
       " 'there',\n",
       " 'were',\n",
       " 'niagara',\n",
       " 'but',\n",
       " 'a',\n",
       " 'cataract',\n",
       " 'of',\n",
       " 'sand',\n",
       " 'would',\n",
       " 'you',\n",
       " 'travel',\n",
       " 'your',\n",
       " 'thousand',\n",
       " 'miles',\n",
       " 'to',\n",
       " 'see',\n",
       " 'it',\n",
       " 'why',\n",
       " 'did',\n",
       " 'the',\n",
       " 'poor',\n",
       " 'poet',\n",
       " 'of',\n",
       " 'tennessee',\n",
       " 'upon',\n",
       " 'suddenly',\n",
       " 'receiving',\n",
       " 'two',\n",
       " 'handfuls',\n",
       " 'of',\n",
       " 'silver',\n",
       " 'deliberate',\n",
       " 'whether',\n",
       " 'to',\n",
       " 'buy',\n",
       " 'him',\n",
       " 'a',\n",
       " 'coat',\n",
       " 'which',\n",
       " 'he',\n",
       " 'sadly',\n",
       " 'needed',\n",
       " 'or',\n",
       " 'invest',\n",
       " 'his',\n",
       " 'money',\n",
       " 'in',\n",
       " 'a',\n",
       " 'pedestrian',\n",
       " 'trip',\n",
       " 'to',\n",
       " 'rockaway',\n",
       " 'beach',\n",
       " 'why',\n",
       " 'is',\n",
       " 'almost',\n",
       " 'every',\n",
       " 'robust',\n",
       " 'healthy',\n",
       " 'boy',\n",
       " 'with',\n",
       " 'a',\n",
       " 'robust',\n",
       " 'healthy',\n",
       " 'soul',\n",
       " 'in',\n",
       " 'him',\n",
       " 'at',\n",
       " 'some',\n",
       " 'time',\n",
       " 'or',\n",
       " 'other',\n",
       " 'crazy',\n",
       " 'to',\n",
       " 'go',\n",
       " 'to',\n",
       " 'sea',\n",
       " 'why',\n",
       " 'upon',\n",
       " 'your',\n",
       " 'first',\n",
       " 'voyage',\n",
       " 'as',\n",
       " 'a',\n",
       " 'passenger',\n",
       " 'did',\n",
       " 'you',\n",
       " 'yourself',\n",
       " 'feel',\n",
       " 'such',\n",
       " 'a',\n",
       " 'mystical',\n",
       " 'vibration',\n",
       " 'when',\n",
       " 'first',\n",
       " 'told',\n",
       " 'that',\n",
       " 'you',\n",
       " 'and',\n",
       " 'your',\n",
       " 'ship',\n",
       " 'were',\n",
       " 'now',\n",
       " 'out',\n",
       " 'of',\n",
       " 'sight',\n",
       " 'of',\n",
       " 'land',\n",
       " 'why',\n",
       " 'did',\n",
       " 'the',\n",
       " 'old',\n",
       " 'persians',\n",
       " 'hold',\n",
       " 'the',\n",
       " 'sea',\n",
       " 'holy',\n",
       " 'why',\n",
       " 'did',\n",
       " 'the',\n",
       " 'greeks',\n",
       " 'give',\n",
       " 'it',\n",
       " 'a',\n",
       " 'separate',\n",
       " 'deity',\n",
       " 'and',\n",
       " 'own',\n",
       " 'brother',\n",
       " 'of',\n",
       " 'jove',\n",
       " 'surely',\n",
       " 'all',\n",
       " 'this',\n",
       " 'is',\n",
       " 'not',\n",
       " 'without',\n",
       " 'meaning',\n",
       " 'and',\n",
       " 'still',\n",
       " 'deeper',\n",
       " 'the',\n",
       " 'meaning',\n",
       " 'of',\n",
       " 'that',\n",
       " 'story',\n",
       " 'of',\n",
       " 'narcissus',\n",
       " 'who',\n",
       " 'because',\n",
       " 'he',\n",
       " 'could',\n",
       " 'not',\n",
       " 'grasp',\n",
       " 'the',\n",
       " 'tormenting',\n",
       " 'mild',\n",
       " 'image',\n",
       " 'he',\n",
       " 'saw',\n",
       " 'in',\n",
       " 'the',\n",
       " 'fountain',\n",
       " 'plunged',\n",
       " 'into',\n",
       " 'it',\n",
       " 'and',\n",
       " 'was',\n",
       " 'drowned',\n",
       " 'but',\n",
       " 'that',\n",
       " 'same',\n",
       " 'image',\n",
       " 'we',\n",
       " 'ourselves',\n",
       " 'see',\n",
       " 'in',\n",
       " 'all',\n",
       " 'rivers',\n",
       " 'and',\n",
       " 'oceans',\n",
       " 'it',\n",
       " 'is',\n",
       " 'the',\n",
       " 'image',\n",
       " 'of',\n",
       " 'the',\n",
       " 'ungraspable',\n",
       " 'phantom',\n",
       " 'of',\n",
       " 'life',\n",
       " 'and',\n",
       " 'this',\n",
       " 'is',\n",
       " 'the',\n",
       " 'key',\n",
       " 'to',\n",
       " 'it',\n",
       " 'all',\n",
       " 'now',\n",
       " 'when',\n",
       " 'i',\n",
       " 'say',\n",
       " 'that',\n",
       " 'i',\n",
       " 'am',\n",
       " 'in',\n",
       " 'the',\n",
       " 'habit',\n",
       " 'of',\n",
       " 'going',\n",
       " 'to',\n",
       " 'sea',\n",
       " 'whenever',\n",
       " 'i',\n",
       " 'begin',\n",
       " 'to',\n",
       " 'grow',\n",
       " 'hazy',\n",
       " 'about',\n",
       " 'the',\n",
       " 'eyes',\n",
       " 'and',\n",
       " ...]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11338"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(tokens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_len = 25 + 1\n",
    "text_sequences = []\n",
    "\n",
    "for i in range(train_len, len(tokens)):\n",
    "    seq = tokens[i - train_len:i]\n",
    "    \n",
    "    text_sequences.append(seq)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(text_sequences)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['me',\n",
       " 'ishmael',\n",
       " 'some',\n",
       " 'years',\n",
       " 'ago',\n",
       " 'never',\n",
       " 'mind',\n",
       " 'how',\n",
       " 'long',\n",
       " 'precisely',\n",
       " 'having',\n",
       " 'little',\n",
       " 'or',\n",
       " 'no',\n",
       " 'money',\n",
       " 'in',\n",
       " 'my',\n",
       " 'purse',\n",
       " 'and',\n",
       " 'nothing',\n",
       " 'particular',\n",
       " 'to',\n",
       " 'interest',\n",
       " 'me',\n",
       " 'on',\n",
       " 'shore']"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_sequences[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.preprocessing.text import Tokenizer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenizer = Tokenizer()\n",
    "tokenizer.fit_on_texts(text_sequences)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "sequences = tokenizer.texts_to_sequences(text_sequences)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([14,\n",
       "  263,\n",
       "  51,\n",
       "  261,\n",
       "  408,\n",
       "  87,\n",
       "  219,\n",
       "  129,\n",
       "  111,\n",
       "  954,\n",
       "  260,\n",
       "  50,\n",
       "  43,\n",
       "  38,\n",
       "  315,\n",
       "  7,\n",
       "  23,\n",
       "  546,\n",
       "  3,\n",
       "  150,\n",
       "  259,\n",
       "  6,\n",
       "  2712,\n",
       "  14,\n",
       "  24,\n",
       "  957],\n",
       " [956,\n",
       "  14,\n",
       "  263,\n",
       "  51,\n",
       "  261,\n",
       "  408,\n",
       "  87,\n",
       "  219,\n",
       "  129,\n",
       "  111,\n",
       "  954,\n",
       "  260,\n",
       "  50,\n",
       "  43,\n",
       "  38,\n",
       "  315,\n",
       "  7,\n",
       "  23,\n",
       "  546,\n",
       "  3,\n",
       "  150,\n",
       "  259,\n",
       "  6,\n",
       "  2712,\n",
       "  14,\n",
       "  24])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#sequences[1], sequences[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{1: 'the',\n",
       " 2: 'a',\n",
       " 3: 'and',\n",
       " 4: 'of',\n",
       " 5: 'i',\n",
       " 6: 'to',\n",
       " 7: 'in',\n",
       " 8: 'it',\n",
       " 9: 'that',\n",
       " 10: 'he',\n",
       " 11: 'his',\n",
       " 12: 'was',\n",
       " 13: 'but',\n",
       " 14: 'me',\n",
       " 15: 'with',\n",
       " 16: 'as',\n",
       " 17: 'at',\n",
       " 18: 'this',\n",
       " 19: 'you',\n",
       " 20: 'is',\n",
       " 21: 'all',\n",
       " 22: 'for',\n",
       " 23: 'my',\n",
       " 24: 'on',\n",
       " 25: 'be',\n",
       " 26: \"'s\",\n",
       " 27: 'not',\n",
       " 28: 'from',\n",
       " 29: 'there',\n",
       " 30: 'one',\n",
       " 31: 'up',\n",
       " 32: 'what',\n",
       " 33: 'him',\n",
       " 34: 'so',\n",
       " 35: 'bed',\n",
       " 36: 'now',\n",
       " 37: 'about',\n",
       " 38: 'no',\n",
       " 39: 'into',\n",
       " 40: 'by',\n",
       " 41: 'were',\n",
       " 42: 'out',\n",
       " 43: 'or',\n",
       " 44: 'harpooneer',\n",
       " 45: 'had',\n",
       " 46: 'then',\n",
       " 47: 'have',\n",
       " 48: 'an',\n",
       " 49: 'upon',\n",
       " 50: 'little',\n",
       " 51: 'some',\n",
       " 52: 'old',\n",
       " 53: 'like',\n",
       " 54: 'if',\n",
       " 55: 'they',\n",
       " 56: 'would',\n",
       " 57: 'do',\n",
       " 58: 'over',\n",
       " 59: 'landlord',\n",
       " 60: 'thought',\n",
       " 61: 'room',\n",
       " 62: 'when',\n",
       " 63: 'could',\n",
       " 64: \"n't\",\n",
       " 65: 'night',\n",
       " 66: 'here',\n",
       " 67: 'head',\n",
       " 68: 'such',\n",
       " 69: 'which',\n",
       " 70: 'man',\n",
       " 71: 'did',\n",
       " 72: 'sea',\n",
       " 73: 'time',\n",
       " 74: 'other',\n",
       " 75: 'very',\n",
       " 76: 'go',\n",
       " 77: 'these',\n",
       " 78: 'more',\n",
       " 79: 'though',\n",
       " 80: 'first',\n",
       " 81: 'sort',\n",
       " 82: 'said',\n",
       " 83: 'last',\n",
       " 84: 'down',\n",
       " 85: 'most',\n",
       " 86: 'been',\n",
       " 87: 'never',\n",
       " 88: 'your',\n",
       " 89: 'them',\n",
       " 90: 'must',\n",
       " 91: 'tell',\n",
       " 92: 'much',\n",
       " 93: 'good',\n",
       " 94: 'see',\n",
       " 95: 'off',\n",
       " 96: 'myself',\n",
       " 97: 'are',\n",
       " 98: 'yet',\n",
       " 99: 'sleep',\n",
       " 100: 'who',\n",
       " 101: 'seemed',\n",
       " 102: 'light',\n",
       " 103: 'way',\n",
       " 104: 'their',\n",
       " 105: 'just',\n",
       " 106: 'being',\n",
       " 107: 'than',\n",
       " 108: 'place',\n",
       " 109: 'queequeg',\n",
       " 110: 'great',\n",
       " 111: 'long',\n",
       " 112: 'before',\n",
       " 113: 'get',\n",
       " 114: 'round',\n",
       " 115: 'where',\n",
       " 116: 'still',\n",
       " 117: 'any',\n",
       " 118: 'too',\n",
       " 119: 'only',\n",
       " 120: 'door',\n",
       " 121: 'can',\n",
       " 122: 'himself',\n",
       " 123: 'heads',\n",
       " 124: 'come',\n",
       " 125: 'ever',\n",
       " 126: 'two',\n",
       " 127: 'enough',\n",
       " 128: 'made',\n",
       " 129: 'how',\n",
       " 130: 'hand',\n",
       " 131: 'same',\n",
       " 132: 'looking',\n",
       " 133: 'something',\n",
       " 134: 'may',\n",
       " 135: \"'\",\n",
       " 136: 'almost',\n",
       " 137: 'say',\n",
       " 138: 'should',\n",
       " 139: 'side',\n",
       " 140: 'why',\n",
       " 141: 'own',\n",
       " 142: 'we',\n",
       " 143: 'new',\n",
       " 144: 'again',\n",
       " 145: 'came',\n",
       " 146: 'arm',\n",
       " 147: 'house',\n",
       " 148: 'away',\n",
       " 149: 'might',\n",
       " 150: 'nothing',\n",
       " 151: 'take',\n",
       " 152: 'towards',\n",
       " 153: 'will',\n",
       " 154: 'under',\n",
       " 155: 'going',\n",
       " 156: 'make',\n",
       " 157: 'whale',\n",
       " 158: 'stood',\n",
       " 159: 'boots',\n",
       " 160: 'ye',\n",
       " 161: 'back',\n",
       " 162: \"'ll\",\n",
       " 163: 'tomahawk',\n",
       " 164: 'part',\n",
       " 165: 'world',\n",
       " 166: 'soon',\n",
       " 167: 'water',\n",
       " 168: 'against',\n",
       " 169: 'those',\n",
       " 170: 'between',\n",
       " 171: 'after',\n",
       " 172: 'whaling',\n",
       " 173: 'lay',\n",
       " 174: 'took',\n",
       " 175: 'half',\n",
       " 176: 'began',\n",
       " 177: 'face',\n",
       " 178: 'streets',\n",
       " 179: 'land',\n",
       " 180: 'better',\n",
       " 181: 'once',\n",
       " 182: 'voyage',\n",
       " 183: 'give',\n",
       " 184: 'rather',\n",
       " 185: 'well',\n",
       " 186: 'however',\n",
       " 187: 'else',\n",
       " 188: 'heard',\n",
       " 189: 'put',\n",
       " 190: 'stop',\n",
       " 191: 'dark',\n",
       " 192: 'went',\n",
       " 193: 'black',\n",
       " 194: 'window',\n",
       " 195: 'cannibal',\n",
       " 196: 'fire',\n",
       " 197: 'every',\n",
       " 198: 'ship',\n",
       " 199: 'stand',\n",
       " 200: 'strange',\n",
       " 201: 'without',\n",
       " 202: 'feet',\n",
       " 203: 'whether',\n",
       " 204: 'because',\n",
       " 205: 'eyes',\n",
       " 206: 'think',\n",
       " 207: 'thinks',\n",
       " 208: 'idea',\n",
       " 209: 'bag',\n",
       " 210: 'nantucket',\n",
       " 211: 'late',\n",
       " 212: 'cold',\n",
       " 213: 'our',\n",
       " 214: 'found',\n",
       " 215: 'full',\n",
       " 216: 'morning',\n",
       " 217: 'sleeping',\n",
       " 218: 'got',\n",
       " 219: 'mind',\n",
       " 220: 'her',\n",
       " 221: 'right',\n",
       " 222: 'its',\n",
       " 223: 'look',\n",
       " 224: 'town',\n",
       " 225: 'south',\n",
       " 226: 'does',\n",
       " 227: 'let',\n",
       " 228: 'set',\n",
       " 229: 'yourself',\n",
       " 230: 'image',\n",
       " 231: 'saw',\n",
       " 232: 'am',\n",
       " 233: 'besides',\n",
       " 234: 'sailor',\n",
       " 235: 'seas',\n",
       " 236: 'rolled',\n",
       " 237: 'till',\n",
       " 238: 'day',\n",
       " 239: 'sign',\n",
       " 240: 'looked',\n",
       " 241: 'hard',\n",
       " 242: 'moment',\n",
       " 243: 'corner',\n",
       " 244: 'entry',\n",
       " 245: 'four',\n",
       " 246: 'wall',\n",
       " 247: 'savage',\n",
       " 248: 'table',\n",
       " 249: 'indeed',\n",
       " 250: 'bench',\n",
       " 251: 'chest',\n",
       " 252: 'while',\n",
       " 253: 'stranger',\n",
       " 254: 'possible',\n",
       " 255: 'feeling',\n",
       " 256: 'floor',\n",
       " 257: 'squares',\n",
       " 258: 'hat',\n",
       " 259: 'particular',\n",
       " 260: 'having',\n",
       " 261: 'years',\n",
       " 262: 'harpoon',\n",
       " 263: 'ishmael',\n",
       " 264: 'whenever',\n",
       " 265: 'mouth',\n",
       " 266: 'high',\n",
       " 267: 'knew',\n",
       " 268: 'men',\n",
       " 269: 'hours',\n",
       " 270: 'green',\n",
       " 271: 'bit',\n",
       " 272: 'within',\n",
       " 273: 'picture',\n",
       " 274: 'told',\n",
       " 275: 'story',\n",
       " 276: 'mean',\n",
       " 277: 'speak',\n",
       " 278: 'order',\n",
       " 279: 'making',\n",
       " 280: 'even',\n",
       " 281: 'perhaps',\n",
       " 282: 'things',\n",
       " 283: 'answer',\n",
       " 284: 'parts',\n",
       " 285: 'wild',\n",
       " 286: 'reason',\n",
       " 287: 'young',\n",
       " 288: 'craft',\n",
       " 289: 'business',\n",
       " 290: 'dead',\n",
       " 291: 'another',\n",
       " 292: 'middle',\n",
       " 293: 'sure',\n",
       " 294: 'candle',\n",
       " 295: 'presently',\n",
       " 296: 'low',\n",
       " 297: 'turned',\n",
       " 298: 'teeth',\n",
       " 299: 'dim',\n",
       " 300: 'euroclydon',\n",
       " 301: 'kept',\n",
       " 302: 'glass',\n",
       " 303: 'afterwards',\n",
       " 304: 'large',\n",
       " 305: 'three',\n",
       " 306: 'telling',\n",
       " 307: 'getting',\n",
       " 308: 'small',\n",
       " 309: 'next',\n",
       " 310: 'seeing',\n",
       " 311: 'sell',\n",
       " 312: 'felt',\n",
       " 313: 'sun',\n",
       " 314: 'e',\n",
       " 315: 'money',\n",
       " 316: 'sail',\n",
       " 317: 'coffin',\n",
       " 318: 'especially',\n",
       " 319: 'street',\n",
       " 320: 'city',\n",
       " 321: 'few',\n",
       " 322: 'previous',\n",
       " 323: 'sight',\n",
       " 324: 'days',\n",
       " 325: 'straight',\n",
       " 326: 'nigh',\n",
       " 327: 'legs',\n",
       " 328: 'try',\n",
       " 329: 'yes',\n",
       " 330: 'unless',\n",
       " 331: 'poor',\n",
       " 332: 'coat',\n",
       " 333: 'passenger',\n",
       " 334: 'taking',\n",
       " 335: 'true',\n",
       " 336: 'thing',\n",
       " 337: 'ai',\n",
       " 338: 'always',\n",
       " 339: 'us',\n",
       " 340: 'really',\n",
       " 341: 'marvellous',\n",
       " 342: 'heaven',\n",
       " 343: 'air',\n",
       " 344: 'far',\n",
       " 345: 'second',\n",
       " 346: 'many',\n",
       " 347: 'has',\n",
       " 348: 'unaccountable',\n",
       " 349: 'grand',\n",
       " 350: 'jolly',\n",
       " 351: 'open',\n",
       " 352: 'shirt',\n",
       " 353: 'cape',\n",
       " 354: 'bedford',\n",
       " 355: 'fine',\n",
       " 356: 'further',\n",
       " 357: 'ice',\n",
       " 358: 'frost',\n",
       " 359: 'foot',\n",
       " 360: 'wide',\n",
       " 361: 'white',\n",
       " 362: 'tall',\n",
       " 363: 'i.',\n",
       " 364: 'wooden',\n",
       " 365: 'worse',\n",
       " 366: 'death',\n",
       " 367: 'mine',\n",
       " 368: 'lazarus',\n",
       " 369: 'keep',\n",
       " 370: 'along',\n",
       " 371: 'hung',\n",
       " 372: 'throwing',\n",
       " 373: 'centre',\n",
       " 374: 'rest',\n",
       " 375: 'fact',\n",
       " 376: 'hair',\n",
       " 377: 'broken',\n",
       " 378: 'kill',\n",
       " 379: 'through',\n",
       " 380: 'chimney',\n",
       " 381: 'fancy',\n",
       " 382: 'bar',\n",
       " 383: 'trying',\n",
       " 384: 'dumplings',\n",
       " 385: 'heavens',\n",
       " 386: 'manner',\n",
       " 387: 'devil',\n",
       " 388: 'together',\n",
       " 389: 'seen',\n",
       " 390: 'deal',\n",
       " 391: 'know',\n",
       " 392: 'skin',\n",
       " 393: 'ca',\n",
       " 394: 'shavings',\n",
       " 395: 'peddling',\n",
       " 396: 'sunday',\n",
       " 397: 'counterpane',\n",
       " 398: 'mat',\n",
       " 399: 'christian',\n",
       " 400: 'commenced',\n",
       " 401: 'thinking',\n",
       " 402: 'similar',\n",
       " 403: 'afraid',\n",
       " 404: 'length',\n",
       " 405: 'idol',\n",
       " 406: 'sabbee',\n",
       " 407: 'waking',\n",
       " 408: 'ago',\n",
       " 409: 'find',\n",
       " 410: 'damp',\n",
       " 411: 'soul',\n",
       " 412: 'strong',\n",
       " 413: 'account',\n",
       " 414: 'sword',\n",
       " 415: 'quietly',\n",
       " 416: 'degree',\n",
       " 417: 'left',\n",
       " 418: 'around',\n",
       " 419: 'fixed',\n",
       " 420: 'ships',\n",
       " 421: 'miles',\n",
       " 422: 'country',\n",
       " 423: 'stream',\n",
       " 424: 'lead',\n",
       " 425: 'american',\n",
       " 426: 'artist',\n",
       " 427: 'each',\n",
       " 428: 'goes',\n",
       " 429: 'deep',\n",
       " 430: 'distant',\n",
       " 431: 'winds',\n",
       " 432: 'blue',\n",
       " 433: 'among',\n",
       " 434: 'suddenly',\n",
       " 435: 'feel',\n",
       " 436: 'meaning',\n",
       " 437: 'phantom',\n",
       " 438: 'life',\n",
       " 439: 'passengers',\n",
       " 440: 'nor',\n",
       " 441: 'kind',\n",
       " 442: 'quite',\n",
       " 443: 'care',\n",
       " 444: 'board',\n",
       " 445: 'somehow',\n",
       " 446: 'broiled',\n",
       " 447: 'mast',\n",
       " 448: 'sense',\n",
       " 449: 'knowing',\n",
       " 450: 'either',\n",
       " 451: 'passed',\n",
       " 452: 'hands',\n",
       " 453: 'paying',\n",
       " 454: 'pay',\n",
       " 455: 'penny',\n",
       " 456: 'sailors',\n",
       " 457: 'exactly',\n",
       " 458: 'short',\n",
       " 459: 'easy',\n",
       " 460: 'portentous',\n",
       " 461: 'island',\n",
       " 462: 'nameless',\n",
       " 463: 'sounds',\n",
       " 464: 'since',\n",
       " 465: 'snow',\n",
       " 466: 'saturday',\n",
       " 467: 'matter',\n",
       " 468: 'red',\n",
       " 469: 'partly',\n",
       " 470: 'ere',\n",
       " 471: 'became',\n",
       " 472: 'meanwhile',\n",
       " 473: 'pocket',\n",
       " 474: 'darkness',\n",
       " 475: 'fish',\n",
       " 476: 'inn',\n",
       " 477: 'watch',\n",
       " 478: 'broad',\n",
       " 479: 'entering',\n",
       " 480: 'ha',\n",
       " 481: 'ashes',\n",
       " 482: 'opened',\n",
       " 483: 'spouter',\n",
       " 484: 'name',\n",
       " 485: 'suppose',\n",
       " 486: 'quiet',\n",
       " 487: 'best',\n",
       " 488: 'tempestuous',\n",
       " 489: 'says',\n",
       " 490: 'thou',\n",
       " 491: 'both',\n",
       " 492: 'occurred',\n",
       " 493: 'dives',\n",
       " 494: 'holding',\n",
       " 495: 'frozen',\n",
       " 496: 'altogether',\n",
       " 497: 'plain',\n",
       " 498: 'whom',\n",
       " 499: 'clean',\n",
       " 500: 'human',\n",
       " 501: 'entered',\n",
       " 502: 'wrinkled',\n",
       " 503: 'shelf',\n",
       " 504: 'jonah',\n",
       " 505: 'blanket',\n",
       " 506: \"goin'\",\n",
       " 507: 'bitter',\n",
       " 508: 'supper',\n",
       " 509: 'sat',\n",
       " 510: 'settle',\n",
       " 511: 'chap',\n",
       " 512: 'help',\n",
       " 513: 'spend',\n",
       " 514: 'landed',\n",
       " 515: 'standing',\n",
       " 516: 'held',\n",
       " 517: 'somewhat',\n",
       " 518: 'sober',\n",
       " 519: 'whole',\n",
       " 520: 'dam',\n",
       " 521: 'brown',\n",
       " 522: 'bulkington',\n",
       " 523: \"o'clock\",\n",
       " 524: 'none',\n",
       " 525: 'coming',\n",
       " 526: \"'ve\",\n",
       " 527: 'wait',\n",
       " 528: 'plane',\n",
       " 529: 'saying',\n",
       " 530: 'grinning',\n",
       " 531: 'placed',\n",
       " 532: 'shouted',\n",
       " 533: 'bedfellow',\n",
       " 534: 'zealand',\n",
       " 535: 'sal',\n",
       " 536: 'wash',\n",
       " 537: 'thrown',\n",
       " 538: 'purplish',\n",
       " 539: 'turn',\n",
       " 540: 'completely',\n",
       " 541: 'fear',\n",
       " 542: 'grego',\n",
       " 543: 'baby',\n",
       " 544: 'slowly',\n",
       " 545: 'civilized',\n",
       " 546: 'purse',\n",
       " 547: 'monkey',\n",
       " 548: 'involuntarily',\n",
       " 549: 'pausing',\n",
       " 550: 'warehouses',\n",
       " 551: 'requires',\n",
       " 552: 'people',\n",
       " 553: 'nearly',\n",
       " 554: 'ocean',\n",
       " 555: 'indian',\n",
       " 556: 'waterward',\n",
       " 557: 'battery',\n",
       " 558: 'noble',\n",
       " 559: 'washed',\n",
       " 560: 'crowds',\n",
       " 561: 'sabbath',\n",
       " 562: 'afternoon',\n",
       " 563: 'thence',\n",
       " 564: 'silent',\n",
       " 565: 'thousands',\n",
       " 566: 'reveries',\n",
       " 567: 'leaning',\n",
       " 568: 'seated',\n",
       " 569: 'bulwarks',\n",
       " 570: 'aloft',\n",
       " 571: 'week',\n",
       " 572: 'plaster',\n",
       " 573: 'gone',\n",
       " 574: 'bound',\n",
       " 575: 'content',\n",
       " 576: 'yonder',\n",
       " 577: 'falling',\n",
       " 578: 'north',\n",
       " 579: 'please',\n",
       " 580: 'ten',\n",
       " 581: 'leaves',\n",
       " 582: 'magic',\n",
       " 583: 'plunged',\n",
       " 584: 'metaphysical',\n",
       " 585: 'chief',\n",
       " 586: 'trunk',\n",
       " 587: 'meadow',\n",
       " 588: 'smoke',\n",
       " 589: 'reaching',\n",
       " 590: 'hill',\n",
       " 591: 'thus',\n",
       " 592: 'pine',\n",
       " 593: 'sighs',\n",
       " 594: 'shepherd',\n",
       " 595: 'june',\n",
       " 596: 'scores',\n",
       " 597: 'thousand',\n",
       " 598: 'sadly',\n",
       " 599: 'robust',\n",
       " 600: 'healthy',\n",
       " 601: 'boy',\n",
       " 602: 'crazy',\n",
       " 603: 'hold',\n",
       " 604: 'holy',\n",
       " 605: 'brother',\n",
       " 606: 'ourselves',\n",
       " 607: 'begin',\n",
       " 608: 'grow',\n",
       " 609: 'inferred',\n",
       " 610: 'needs',\n",
       " 611: \"don't\",\n",
       " 612: 'themselves',\n",
       " 613: 'commodore',\n",
       " 614: 'captain',\n",
       " 615: 'cook',\n",
       " 616: 'glory',\n",
       " 617: 'whatsoever',\n",
       " 618: 'confess',\n",
       " 619: 'officer',\n",
       " 620: 'respectfully',\n",
       " 621: 'horse',\n",
       " 622: 'huge',\n",
       " 623: 'houses',\n",
       " 624: 'forecastle',\n",
       " 625: 'jump',\n",
       " 626: 'spar',\n",
       " 627: 'particularly',\n",
       " 628: 'putting',\n",
       " 629: 'tar',\n",
       " 630: 'schoolmaster',\n",
       " 631: 'boys',\n",
       " 632: 'transition',\n",
       " 633: 'grin',\n",
       " 634: 'bear',\n",
       " 635: 'hunks',\n",
       " 636: 'sweep',\n",
       " 637: 'weighed',\n",
       " 638: 'anything',\n",
       " 639: 'less',\n",
       " 640: 'thump',\n",
       " 641: 'point',\n",
       " 642: 'view',\n",
       " 643: 'single',\n",
       " 644: 'difference',\n",
       " 645: 'act',\n",
       " 646: 'uncomfortable',\n",
       " 647: 'compare',\n",
       " 648: 'earthly',\n",
       " 649: 'enter',\n",
       " 650: 'deck',\n",
       " 651: 'quarter',\n",
       " 652: 'leaders',\n",
       " 653: 'smelt',\n",
       " 654: 'fates',\n",
       " 655: 'doubtless',\n",
       " 656: 'formed',\n",
       " 657: 'run',\n",
       " 658: 'stage',\n",
       " 659: 'shabby',\n",
       " 660: 'others',\n",
       " 661: 'circumstances',\n",
       " 662: 'motives',\n",
       " 663: 'various',\n",
       " 664: 'mysterious',\n",
       " 665: 'curiosity',\n",
       " 666: 'tormented',\n",
       " 667: 'everlasting',\n",
       " 668: 'wonder',\n",
       " 669: 'purpose',\n",
       " 670: 'floated',\n",
       " 671: 'stuffed',\n",
       " 672: 'horn',\n",
       " 673: 'arrived',\n",
       " 674: 'offer',\n",
       " 675: 'following',\n",
       " 676: 'embark',\n",
       " 677: 'pleased',\n",
       " 678: 'original',\n",
       " 679: 'stranded',\n",
       " 680: 'leviathan',\n",
       " 681: 'whales',\n",
       " 682: 'nay',\n",
       " 683: 'dismal',\n",
       " 684: 'wherever',\n",
       " 685: 'dreary',\n",
       " 686: 'crossed',\n",
       " 687: 'expensive',\n",
       " 688: 'bright',\n",
       " 689: 'windows',\n",
       " 690: 'packed',\n",
       " 691: 'inches',\n",
       " 692: 'thick',\n",
       " 693: 'hear',\n",
       " 694: 'tinkling',\n",
       " 695: 'glasses',\n",
       " 696: 'blackness',\n",
       " 697: 'moving',\n",
       " 698: 'hour',\n",
       " 699: 'proved',\n",
       " 700: 'meant',\n",
       " 701: 'public',\n",
       " 702: 'box',\n",
       " 703: 'flying',\n",
       " 704: 'harpoons',\n",
       " 705: 'trap',\n",
       " 706: 'loud',\n",
       " 707: 'voice',\n",
       " 708: 'sitting',\n",
       " 709: 'beyond',\n",
       " 710: 'negro',\n",
       " 711: 'creaking',\n",
       " 712: 'swinging',\n",
       " 713: 'painting',\n",
       " 714: 'representing',\n",
       " 715: 'connexion',\n",
       " 716: 'peter',\n",
       " 717: 'itself',\n",
       " 718: 'carted',\n",
       " 719: 'burnt',\n",
       " 720: 'spot',\n",
       " 721: 'queer',\n",
       " 722: 'gable',\n",
       " 723: 'ended',\n",
       " 724: 'sharp',\n",
       " 725: 'wind',\n",
       " 726: 'howling',\n",
       " 727: 'nevertheless',\n",
       " 728: 'called',\n",
       " 729: 'outside',\n",
       " 730: 'passage',\n",
       " 731: 'body',\n",
       " 732: 'curbstone',\n",
       " 733: 'corn',\n",
       " 734: 'pooh',\n",
       " 735: 'northern',\n",
       " 736: 'lights',\n",
       " 737: 'summer',\n",
       " 738: 'lengthwise',\n",
       " 739: 'gods',\n",
       " 740: 'lie',\n",
       " 741: 'plenty',\n",
       " 742: 'study',\n",
       " 743: 'series',\n",
       " 744: 'arrive',\n",
       " 745: 'shadows',\n",
       " 746: 'dint',\n",
       " 747: 'conclusion',\n",
       " 748: 'confounded',\n",
       " 749: 'limber',\n",
       " 750: 'truly',\n",
       " 751: 'drive',\n",
       " 752: 'unimaginable',\n",
       " 753: 'midnight',\n",
       " 754: 'unnatural',\n",
       " 755: 'breaking',\n",
       " 756: 'design',\n",
       " 757: 'opposite',\n",
       " 758: 'monstrous',\n",
       " 759: 'knots',\n",
       " 760: 'vast',\n",
       " 761: 'handle',\n",
       " 762: 'wondered',\n",
       " 763: 'mixed',\n",
       " 764: 'deformed',\n",
       " 765: 'flung',\n",
       " 766: 'iron',\n",
       " 767: 'arched',\n",
       " 768: 'cut',\n",
       " 769: 'times',\n",
       " 770: 'beneath',\n",
       " 771: 'covered',\n",
       " 772: 'gathered',\n",
       " 773: 'stands',\n",
       " 774: 'rude',\n",
       " 775: 'bone',\n",
       " 776: 'abominable',\n",
       " 777: 'measure',\n",
       " 778: 'seamen',\n",
       " 779: 'skrimshander',\n",
       " 780: 'sought',\n",
       " 781: 'added',\n",
       " 782: 'forehead',\n",
       " 783: 'objections',\n",
       " 784: \"'d\",\n",
       " 785: 'used',\n",
       " 786: 'depend',\n",
       " 787: 'decent',\n",
       " 788: 'want',\n",
       " 789: 'ready',\n",
       " 790: 'working',\n",
       " 791: 'space',\n",
       " 792: 'lips',\n",
       " 793: 'fingers',\n",
       " 794: 'fare',\n",
       " 795: 'fellow',\n",
       " 796: 'nightmare',\n",
       " 797: 'nt',\n",
       " 798: 'complexioned',\n",
       " 799: 'eats',\n",
       " 800: \"'em\",\n",
       " 801: 'afore',\n",
       " 802: 'resolved',\n",
       " 803: 'noise',\n",
       " 804: 'offing',\n",
       " 805: 'shaggy',\n",
       " 806: 'woollen',\n",
       " 807: 'stiff',\n",
       " 808: 'labrador',\n",
       " 809: 'wake',\n",
       " 810: 'bad',\n",
       " 811: 'mounted',\n",
       " 812: 'generally',\n",
       " 813: 'shipmates',\n",
       " 814: 'become',\n",
       " 815: 'height',\n",
       " 816: 'seem',\n",
       " 817: 'plan',\n",
       " 818: 'private',\n",
       " 819: 'unknown',\n",
       " 820: 'apartment',\n",
       " 821: 'hammock',\n",
       " 822: 'linen',\n",
       " 823: 'home',\n",
       " 824: 'hole',\n",
       " 825: 'mattress',\n",
       " 826: 'planing',\n",
       " 827: 'knot',\n",
       " 828: 'near',\n",
       " 829: 'sake',\n",
       " 830: 'chair',\n",
       " 831: 'narrow',\n",
       " 832: 'leaving',\n",
       " 833: 'interval',\n",
       " 834: 'met',\n",
       " 835: 'inside',\n",
       " 836: 'violent',\n",
       " 837: 'ones',\n",
       " 838: 'comprehension',\n",
       " 839: 'bird',\n",
       " 840: 'airley',\n",
       " 841: 'airth',\n",
       " 842: 'engaged',\n",
       " 843: 'whittling',\n",
       " 844: 'guess',\n",
       " 845: 'done',\n",
       " 846: 'break',\n",
       " 847: 'broke',\n",
       " 848: 'understand',\n",
       " 849: 'certain',\n",
       " 850: 'demand',\n",
       " 851: 'selling',\n",
       " 852: 'sir',\n",
       " 853: 'string',\n",
       " 854: 'mystery',\n",
       " 855: 'showed',\n",
       " 856: 'dangerous',\n",
       " 857: 'flukes',\n",
       " 858: 'slept',\n",
       " 859: 'big',\n",
       " 860: 'sam',\n",
       " 861: 'lighted',\n",
       " 862: 'wo',\n",
       " 863: 'stairs',\n",
       " 864: 'placing',\n",
       " 865: 'double',\n",
       " 866: 'eyeing',\n",
       " 867: 'belonging',\n",
       " 868: 'papered',\n",
       " 869: 'parcel',\n",
       " 870: 'tried',\n",
       " 871: 'satisfactory',\n",
       " 872: 'concerning',\n",
       " 873: 'edges',\n",
       " 874: 'stuck',\n",
       " 875: 'gave',\n",
       " 876: 'neck',\n",
       " 877: 'sleeves',\n",
       " 878: 'undressed',\n",
       " 879: 'remembering',\n",
       " 880: 'jumped',\n",
       " 881: 'pantaloons',\n",
       " 882: 'blowing',\n",
       " 883: 'doze',\n",
       " 884: 'pretty',\n",
       " 885: 'heavy',\n",
       " 886: 'save',\n",
       " 887: 'infernal',\n",
       " 888: 'peddler',\n",
       " 889: 'yellow',\n",
       " 890: 'colour',\n",
       " 891: 'dreadfully',\n",
       " 892: 'sticking',\n",
       " 893: 'cheeks',\n",
       " 894: 'truth',\n",
       " 895: 'remembered',\n",
       " 896: 'whaleman',\n",
       " 897: 'tattooed',\n",
       " 898: 'concluded',\n",
       " 899: 'lying',\n",
       " 900: 'tanning',\n",
       " 901: 'hot',\n",
       " 902: 'produced',\n",
       " 903: 'beaver',\n",
       " 904: 'singing',\n",
       " 905: 'bolted',\n",
       " 906: 'arms',\n",
       " 907: 'running',\n",
       " 908: 'curious',\n",
       " 909: 'hunch',\n",
       " 910: 'congo',\n",
       " 911: 'ill',\n",
       " 912: 'takes',\n",
       " 913: 'biscuit',\n",
       " 914: 'top',\n",
       " 915: 'lamp',\n",
       " 916: 'succeeded',\n",
       " 917: 'polite',\n",
       " 918: 'guttural',\n",
       " 919: 'pagan',\n",
       " 920: 'spell',\n",
       " 921: 'tobacco',\n",
       " 922: 'grunt',\n",
       " 923: 'whatever',\n",
       " 924: 'ee',\n",
       " 925: 'horrid',\n",
       " 926: 'pipe',\n",
       " 927: 'smoking',\n",
       " 928: 'complied',\n",
       " 929: 'patchwork',\n",
       " 930: 'figure',\n",
       " 931: 'shade',\n",
       " 932: 'quilt',\n",
       " 933: 'hugging',\n",
       " 934: 'sensations',\n",
       " 935: 'explain',\n",
       " 936: 'remember',\n",
       " 937: 'circumstance',\n",
       " 938: 'reality',\n",
       " 939: 'stepmother',\n",
       " 940: 'sixteen',\n",
       " 941: 'abed',\n",
       " 942: 'troubled',\n",
       " 943: 'supernatural',\n",
       " 944: 'ages',\n",
       " 945: 'awful',\n",
       " 946: 'consciousness',\n",
       " 947: 'observing',\n",
       " 948: 'creature',\n",
       " 949: 'dress',\n",
       " 950: 'watching',\n",
       " 951: 'probably',\n",
       " 952: 'toilet',\n",
       " 953: 'wrapped',\n",
       " 954: 'precisely',\n",
       " 955: 'jacket',\n",
       " 956: 'call',\n",
       " 957: 'shore',\n",
       " 958: 'watery',\n",
       " 959: 'driving',\n",
       " 960: 'spleen',\n",
       " 961: 'regulating',\n",
       " 962: 'circulation',\n",
       " 963: 'growing',\n",
       " 964: 'grim',\n",
       " 965: 'drizzly',\n",
       " 966: 'november',\n",
       " 967: 'bringing',\n",
       " 968: 'rear',\n",
       " 969: 'funeral',\n",
       " 970: 'meet',\n",
       " 971: 'hypos',\n",
       " 972: 'upper',\n",
       " 973: 'moral',\n",
       " 974: 'principle',\n",
       " 975: 'prevent',\n",
       " 976: 'deliberately',\n",
       " 977: 'stepping',\n",
       " 978: 'methodically',\n",
       " 979: 'knocking',\n",
       " 980: 'hats',\n",
       " 981: 'substitute',\n",
       " 982: 'pistol',\n",
       " 983: 'ball',\n",
       " 984: 'philosophical',\n",
       " 985: 'flourish',\n",
       " 986: 'cato',\n",
       " 987: 'throws',\n",
       " 988: 'surprising',\n",
       " 989: 'cherish',\n",
       " 990: 'feelings',\n",
       " 991: 'insular',\n",
       " 992: 'manhattoes',\n",
       " 993: 'belted',\n",
       " 994: 'wharves',\n",
       " 995: 'isles',\n",
       " 996: 'coral',\n",
       " 997: 'reefs',\n",
       " 998: 'commerce',\n",
       " 999: 'surrounds',\n",
       " 1000: 'surf',\n",
       " ...}"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.index_word"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "956 : call\n",
      "14 : me\n",
      "263 : ishmael\n",
      "51 : some\n",
      "261 : years\n",
      "408 : ago\n",
      "87 : never\n",
      "219 : mind\n",
      "129 : how\n",
      "111 : long\n",
      "954 : precisely\n",
      "260 : having\n",
      "50 : little\n",
      "43 : or\n",
      "38 : no\n",
      "315 : money\n",
      "7 : in\n",
      "23 : my\n",
      "546 : purse\n",
      "3 : and\n",
      "150 : nothing\n",
      "259 : particular\n",
      "6 : to\n",
      "2712 : interest\n",
      "14 : me\n",
      "24 : on\n"
     ]
    }
   ],
   "source": [
    "for i in sequences[0]:\n",
    "    print(f'{i} : {tokenizer.index_word[i]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OrderedDict([('call', 27),\n",
       "             ('me', 2471),\n",
       "             ('ishmael', 133),\n",
       "             ('some', 758),\n",
       "             ('years', 135),\n",
       "             ('ago', 84),\n",
       "             ('never', 449),\n",
       "             ('mind', 164),\n",
       "             ('how', 321),\n",
       "             ('long', 374),\n",
       "             ('precisely', 37),\n",
       "             ('having', 142),\n",
       "             ('little', 767),\n",
       "             ('or', 950),\n",
       "             ('no', 1003),\n",
       "             ('money', 120),\n",
       "             ('in', 5647),\n",
       "             ('my', 1786),\n",
       "             ('purse', 71),\n",
       "             ('and', 9646),\n",
       "             ('nothing', 281),\n",
       "             ('particular', 152),\n",
       "             ('to', 6497),\n",
       "             ('interest', 24),\n",
       "             ('on', 1716),\n",
       "             ('shore', 26),\n",
       "             ('i', 7150),\n",
       "             ('thought', 676),\n",
       "             ('would', 702),\n",
       "             ('sail', 104),\n",
       "             ('about', 1014),\n",
       "             ('a', 10377),\n",
       "             ('see', 416),\n",
       "             ('the', 15540),\n",
       "             ('watery', 26),\n",
       "             ('part', 234),\n",
       "             ('of', 8287),\n",
       "             ('world', 234),\n",
       "             ('it', 4238),\n",
       "             ('is', 1950),\n",
       "             ('way', 390),\n",
       "             ('have', 806),\n",
       "             ('driving', 26),\n",
       "             ('off', 416),\n",
       "             ('spleen', 26),\n",
       "             ('regulating', 26),\n",
       "             ('circulation', 26),\n",
       "             ('whenever', 130),\n",
       "             ('find', 78),\n",
       "             ('myself', 416),\n",
       "             ('growing', 26),\n",
       "             ('grim', 26),\n",
       "             ('mouth', 130),\n",
       "             ('damp', 78),\n",
       "             ('drizzly', 26),\n",
       "             ('november', 26),\n",
       "             ('soul', 78),\n",
       "             ('involuntarily', 52),\n",
       "             ('pausing', 52),\n",
       "             ('before', 364),\n",
       "             ('coffin', 104),\n",
       "             ('warehouses', 52),\n",
       "             ('bringing', 26),\n",
       "             ('up', 1237),\n",
       "             ('rear', 26),\n",
       "             ('every', 182),\n",
       "             ('funeral', 26),\n",
       "             ('meet', 26),\n",
       "             ('especially', 104),\n",
       "             ('hypos', 26),\n",
       "             ('get', 364),\n",
       "             ('such', 572),\n",
       "             ('an', 806),\n",
       "             ('upper', 26),\n",
       "             ('hand', 312),\n",
       "             ('that', 3770),\n",
       "             ('requires', 52),\n",
       "             ('strong', 78),\n",
       "             ('moral', 26),\n",
       "             ('principle', 26),\n",
       "             ('prevent', 26),\n",
       "             ('from', 1508),\n",
       "             ('deliberately', 26),\n",
       "             ('stepping', 26),\n",
       "             ('into', 988),\n",
       "             ('street', 104),\n",
       "             ('methodically', 26),\n",
       "             ('knocking', 26),\n",
       "             ('people', 52),\n",
       "             (\"'s\", 1691),\n",
       "             ('hats', 26),\n",
       "             ('then', 832),\n",
       "             ('account', 78),\n",
       "             ('high', 130),\n",
       "             ('time', 520),\n",
       "             ('sea', 546),\n",
       "             ('as', 2366),\n",
       "             ('soon', 234),\n",
       "             ('can', 338),\n",
       "             ('this', 2158),\n",
       "             ('substitute', 26),\n",
       "             ('for', 1820),\n",
       "             ('pistol', 26),\n",
       "             ('ball', 26),\n",
       "             ('with', 2392),\n",
       "             ('philosophical', 26),\n",
       "             ('flourish', 26),\n",
       "             ('cato', 26),\n",
       "             ('throws', 26),\n",
       "             ('himself', 338),\n",
       "             ('upon', 780),\n",
       "             ('his', 3139),\n",
       "             ('sword', 78),\n",
       "             ('quietly', 78),\n",
       "             ('take', 260),\n",
       "             ('ship', 182),\n",
       "             ('there', 1456),\n",
       "             ('surprising', 26),\n",
       "             ('if', 728),\n",
       "             ('they', 728),\n",
       "             ('but', 2652),\n",
       "             ('knew', 130),\n",
       "             ('almost', 286),\n",
       "             ('all', 1872),\n",
       "             ('men', 130),\n",
       "             ('their', 390),\n",
       "             ('degree', 78),\n",
       "             ('other', 494),\n",
       "             ('cherish', 26),\n",
       "             ('very', 494),\n",
       "             ('nearly', 52),\n",
       "             ('same', 312),\n",
       "             ('feelings', 26),\n",
       "             ('towards', 260),\n",
       "             ('ocean', 52),\n",
       "             ('now', 1040),\n",
       "             ('your', 442),\n",
       "             ('insular', 26),\n",
       "             ('city', 104),\n",
       "             ('manhattoes', 26),\n",
       "             ('belted', 26),\n",
       "             ('round', 364),\n",
       "             ('by', 962),\n",
       "             ('wharves', 26),\n",
       "             ('indian', 52),\n",
       "             ('isles', 26),\n",
       "             ('coral', 26),\n",
       "             ('reefs', 26),\n",
       "             ('commerce', 26),\n",
       "             ('surrounds', 26),\n",
       "             ('her', 156),\n",
       "             ('surf', 26),\n",
       "             ('right', 156),\n",
       "             ('left', 78),\n",
       "             ('streets', 208),\n",
       "             ('you', 2158),\n",
       "             ('waterward', 52),\n",
       "             ('its', 156),\n",
       "             ('extreme', 26),\n",
       "             ('downtown', 26),\n",
       "             ('battery', 52),\n",
       "             ('where', 364),\n",
       "             ('noble', 52),\n",
       "             ('mole', 26),\n",
       "             ('washed', 52),\n",
       "             ('waves', 26),\n",
       "             ('cooled', 26),\n",
       "             ('breezes', 26),\n",
       "             ('which', 572),\n",
       "             ('few', 104),\n",
       "             ('hours', 130),\n",
       "             ('previous', 104),\n",
       "             ('were', 962),\n",
       "             ('out', 956),\n",
       "             ('sight', 104),\n",
       "             ('land', 208),\n",
       "             ('look', 156),\n",
       "             ('at', 2184),\n",
       "             ('crowds', 52),\n",
       "             ('water', 234),\n",
       "             ('gazers', 26),\n",
       "             ('circumambulate', 26),\n",
       "             ('dreamy', 26),\n",
       "             ('sabbath', 52),\n",
       "             ('afternoon', 52),\n",
       "             ('go', 494),\n",
       "             ('corlears', 26),\n",
       "             ('hook', 26),\n",
       "             ('coenties', 26),\n",
       "             ('slip', 26),\n",
       "             ('thence', 52),\n",
       "             ('whitehall', 26),\n",
       "             ('northward', 26),\n",
       "             ('what', 1118),\n",
       "             ('do', 702),\n",
       "             ('see?--posted', 26),\n",
       "             ('like', 732),\n",
       "             ('silent', 52),\n",
       "             ('sentinels', 26),\n",
       "             ('around', 78),\n",
       "             ('town', 156),\n",
       "             ('stand', 182),\n",
       "             ('thousands', 52),\n",
       "             ('mortal', 26),\n",
       "             ('fixed', 78),\n",
       "             ('reveries', 52),\n",
       "             ('leaning', 52),\n",
       "             ('against', 234),\n",
       "             ('spiles', 26),\n",
       "             ('seated', 52),\n",
       "             ('pier', 26),\n",
       "             ('heads', 338),\n",
       "             ('looking', 312),\n",
       "             ('over', 702),\n",
       "             ('bulwarks', 52),\n",
       "             ('ships', 78),\n",
       "             ('china', 26),\n",
       "             ('aloft', 52),\n",
       "             ('rigging', 26),\n",
       "             ('striving', 26),\n",
       "             ('still', 364),\n",
       "             ('better', 208),\n",
       "             ('seaward', 26),\n",
       "             ('peep', 26),\n",
       "             ('these', 494),\n",
       "             ('are', 416),\n",
       "             ('landsmen', 26),\n",
       "             ('week', 52),\n",
       "             ('days', 104),\n",
       "             ('pent', 26),\n",
       "             ('lath', 26),\n",
       "             ('plaster', 52),\n",
       "             ('tied', 26),\n",
       "             ('counters', 26),\n",
       "             ('nailed', 26),\n",
       "             ('benches', 26),\n",
       "             ('clinched', 26),\n",
       "             ('desks', 26),\n",
       "             ('green', 130),\n",
       "             ('fields', 26),\n",
       "             ('gone', 52),\n",
       "             ('here', 598),\n",
       "             ('come', 338),\n",
       "             ('more', 494),\n",
       "             ('pacing', 26),\n",
       "             ('straight', 104),\n",
       "             ('seemingly', 26),\n",
       "             ('bound', 52),\n",
       "             ('dive', 26),\n",
       "             ('strange', 182),\n",
       "             ('will', 260),\n",
       "             ('content', 52),\n",
       "             ('them', 442),\n",
       "             ('extremest', 26),\n",
       "             ('limit', 26),\n",
       "             ('loitering', 26),\n",
       "             ('under', 260),\n",
       "             ('shady', 26),\n",
       "             ('lee', 26),\n",
       "             ('yonder', 52),\n",
       "             ('not', 1534),\n",
       "             ('suffice', 26),\n",
       "             ('must', 442),\n",
       "             ('just', 390),\n",
       "             ('nigh', 104),\n",
       "             ('possibly', 26),\n",
       "             ('without', 182),\n",
       "             ('falling', 52),\n",
       "             ('miles', 78),\n",
       "             ('leagues', 26),\n",
       "             ('inlanders', 26),\n",
       "             ('lanes', 26),\n",
       "             ('alleys', 26),\n",
       "             ('avenues', 26),\n",
       "             ('north', 52),\n",
       "             ('east', 26),\n",
       "             ('south', 156),\n",
       "             ('west', 26),\n",
       "             ('yet', 416),\n",
       "             ('unite', 26),\n",
       "             ('tell', 442),\n",
       "             ('does', 156),\n",
       "             ('magnetic', 26),\n",
       "             ('virtue', 26),\n",
       "             ('needles', 26),\n",
       "             ('compasses', 26),\n",
       "             ('those', 234),\n",
       "             ('attract', 26),\n",
       "             ('thither', 26),\n",
       "             ('once', 208),\n",
       "             ('say', 286),\n",
       "             ('country', 78),\n",
       "             ('lakes', 26),\n",
       "             ('any', 364),\n",
       "             ('path', 26),\n",
       "             ('please', 52),\n",
       "             ('ten', 52),\n",
       "             ('one', 1300),\n",
       "             ('carries', 26),\n",
       "             ('down', 468),\n",
       "             ('dale', 26),\n",
       "             ('leaves', 52),\n",
       "             ('pool', 26),\n",
       "             ('stream', 78),\n",
       "             ('magic', 52),\n",
       "             ('let', 156),\n",
       "             ('most', 468),\n",
       "             ('absent', 26),\n",
       "             ('minded', 26),\n",
       "             ('be', 1716),\n",
       "             ('plunged', 52),\n",
       "             ('deepest', 26),\n",
       "             ('man', 572),\n",
       "             ('legs', 104),\n",
       "             ('set', 156),\n",
       "             ('feet', 182),\n",
       "             ('going', 260),\n",
       "             ('he', 3247),\n",
       "             ('infallibly', 26),\n",
       "             ('lead', 78),\n",
       "             ('region', 26),\n",
       "             ('should', 286),\n",
       "             ('ever', 338),\n",
       "             ('athirst', 26),\n",
       "             ('great', 376),\n",
       "             ('american', 78),\n",
       "             ('desert', 26),\n",
       "             ('try', 104),\n",
       "             ('experiment', 26),\n",
       "             ('caravan', 26),\n",
       "             ('happen', 26),\n",
       "             ('supplied', 26),\n",
       "             ('metaphysical', 52),\n",
       "             ('professor', 26),\n",
       "             ('yes', 104),\n",
       "             ('knows', 26),\n",
       "             ('meditation', 26),\n",
       "             ('wedded', 26),\n",
       "             ('artist', 78),\n",
       "             ('desires', 26),\n",
       "             ('paint', 26),\n",
       "             ('dreamiest', 26),\n",
       "             ('shadiest', 26),\n",
       "             ('quietest', 26),\n",
       "             ('enchanting', 26),\n",
       "             ('bit', 130),\n",
       "             ('romantic', 26),\n",
       "             ('landscape', 26),\n",
       "             ('valley', 26),\n",
       "             ('saco', 26),\n",
       "             ('chief', 52),\n",
       "             ('element', 26),\n",
       "             ('employs', 26),\n",
       "             ('trees', 26),\n",
       "             ('each', 78),\n",
       "             ('hollow', 26),\n",
       "             ('trunk', 52),\n",
       "             ('hermit', 26),\n",
       "             ('crucifix', 26),\n",
       "             ('within', 130),\n",
       "             ('sleeps', 26),\n",
       "             ('meadow', 52),\n",
       "             ('sleep', 416),\n",
       "             ('cattle', 26),\n",
       "             ('cottage', 26),\n",
       "             ('goes', 78),\n",
       "             ('sleepy', 26),\n",
       "             ('smoke', 52),\n",
       "             ('deep', 78),\n",
       "             ('distant', 78),\n",
       "             ('woodlands', 26),\n",
       "             ('winds', 78),\n",
       "             ('mazy', 26),\n",
       "             ('reaching', 52),\n",
       "             ('overlapping', 26),\n",
       "             ('spurs', 26),\n",
       "             ('mountains', 26),\n",
       "             ('bathed', 26),\n",
       "             ('hill', 52),\n",
       "             ('side', 286),\n",
       "             ('blue', 78),\n",
       "             ('though', 494),\n",
       "             ('picture', 130),\n",
       "             ('lies', 26),\n",
       "             ('thus', 52),\n",
       "             ('tranced', 26),\n",
       "             ('pine', 52),\n",
       "             ('tree', 26),\n",
       "             ('shakes', 26),\n",
       "             ('sighs', 52),\n",
       "             ('shepherd', 52),\n",
       "             ('head', 598),\n",
       "             ('vain', 26),\n",
       "             ('unless', 104),\n",
       "             ('eye', 26),\n",
       "             ('him', 1092),\n",
       "             ('visit', 26),\n",
       "             ('prairies', 26),\n",
       "             ('june', 52),\n",
       "             ('when', 650),\n",
       "             ('scores', 52),\n",
       "             ('wade', 26),\n",
       "             ('knee', 26),\n",
       "             ('among', 78),\n",
       "             ('tiger', 26),\n",
       "             ('lilies', 26),\n",
       "             ('charm', 26),\n",
       "             ('wanting?--water', 26),\n",
       "             ('drop', 26),\n",
       "             ('niagara', 26),\n",
       "             ('cataract', 26),\n",
       "             ('sand', 26),\n",
       "             ('travel', 26),\n",
       "             ('thousand', 52),\n",
       "             ('why', 286),\n",
       "             ('did', 572),\n",
       "             ('poor', 104),\n",
       "             ('poet', 26),\n",
       "             ('tennessee', 26),\n",
       "             ('suddenly', 78),\n",
       "             ('receiving', 26),\n",
       "             ('two', 338),\n",
       "             ('handfuls', 26),\n",
       "             ('silver', 26),\n",
       "             ('deliberate', 26),\n",
       "             ('whether', 182),\n",
       "             ('buy', 26),\n",
       "             ('coat', 104),\n",
       "             ('sadly', 52),\n",
       "             ('needed', 26),\n",
       "             ('invest', 26),\n",
       "             ('pedestrian', 26),\n",
       "             ('trip', 26),\n",
       "             ('rockaway', 26),\n",
       "             ('beach', 26),\n",
       "             ('robust', 52),\n",
       "             ('healthy', 52),\n",
       "             ('boy', 52),\n",
       "             ('crazy', 52),\n",
       "             ('first', 494),\n",
       "             ('voyage', 208),\n",
       "             ('passenger', 104),\n",
       "             ('yourself', 156),\n",
       "             ('feel', 78),\n",
       "             ('mystical', 26),\n",
       "             ('vibration', 26),\n",
       "             ('told', 130),\n",
       "             ('old', 754),\n",
       "             ('persians', 26),\n",
       "             ('hold', 52),\n",
       "             ('holy', 52),\n",
       "             ('greeks', 26),\n",
       "             ('give', 208),\n",
       "             ('separate', 26),\n",
       "             ('deity', 26),\n",
       "             ('own', 286),\n",
       "             ('brother', 52),\n",
       "             ('jove', 26),\n",
       "             ('surely', 26),\n",
       "             ('meaning', 78),\n",
       "             ('deeper', 26),\n",
       "             ('story', 130),\n",
       "             ('narcissus', 26),\n",
       "             ('who', 416),\n",
       "             ('because', 182),\n",
       "             ('could', 650),\n",
       "             ('grasp', 26),\n",
       "             ('tormenting', 26),\n",
       "             ('mild', 26),\n",
       "             ('image', 156),\n",
       "             ('saw', 156),\n",
       "             ('fountain', 26),\n",
       "             ('was', 2886),\n",
       "             ('drowned', 26),\n",
       "             ('we', 286),\n",
       "             ('ourselves', 52),\n",
       "             ('rivers', 26),\n",
       "             ('oceans', 26),\n",
       "             ('ungraspable', 26),\n",
       "             ('phantom', 78),\n",
       "             ('life', 78),\n",
       "             ('key', 26),\n",
       "             ('am', 156),\n",
       "             ('habit', 26),\n",
       "             ('begin', 52),\n",
       "             ('grow', 52),\n",
       "             ('hazy', 26),\n",
       "             ('eyes', 182),\n",
       "             ('conscious', 26),\n",
       "             ('lungs', 26),\n",
       "             ('mean', 130),\n",
       "             ('inferred', 52),\n",
       "             ('needs', 52),\n",
       "             ('rag', 26),\n",
       "             ('something', 312),\n",
       "             ('besides', 156),\n",
       "             ('passengers', 78),\n",
       "             ('sick', 26),\n",
       "             ('quarrelsome', 26),\n",
       "             (\"don't\", 52),\n",
       "             ('nights', 26),\n",
       "             ('enjoy', 26),\n",
       "             ('themselves', 52),\n",
       "             ('much', 442),\n",
       "             ('general', 26),\n",
       "             ('thing;--no', 26),\n",
       "             ('nor', 78),\n",
       "             ('salt', 26),\n",
       "             ('commodore', 52),\n",
       "             ('captain', 52),\n",
       "             ('cook', 52),\n",
       "             ('abandon', 26),\n",
       "             ('glory', 52),\n",
       "             ('distinction', 26),\n",
       "             ('offices', 26),\n",
       "             ('abominate', 26),\n",
       "             ('honourable', 26),\n",
       "             ('respectable', 26),\n",
       "             ('toils', 26),\n",
       "             ('trials', 26),\n",
       "             ('tribulations', 26),\n",
       "             ('kind', 78),\n",
       "             ('whatsoever', 52),\n",
       "             ('quite', 78),\n",
       "             ('care', 78),\n",
       "             ('taking', 104),\n",
       "             ('barques', 26),\n",
       "             ('brigs', 26),\n",
       "             ('schooners', 26),\n",
       "             ('cook,--though', 26),\n",
       "             ('confess', 52),\n",
       "             ('considerable', 26),\n",
       "             ('being', 390),\n",
       "             ('sort', 494),\n",
       "             ('officer', 52),\n",
       "             ('board', 78),\n",
       "             ('somehow', 78),\n",
       "             ('fancied', 26),\n",
       "             ('broiling', 26),\n",
       "             ('fowls;--though', 26),\n",
       "             ('broiled', 78),\n",
       "             ('judiciously', 26),\n",
       "             ('buttered', 26),\n",
       "             ('judgmatically', 26),\n",
       "             ('salted', 26),\n",
       "             ('peppered', 26),\n",
       "             ('speak', 130),\n",
       "             ('respectfully', 52),\n",
       "             ('reverentially', 26),\n",
       "             ('fowl', 26),\n",
       "             ('than', 390),\n",
       "             ('idolatrous', 26),\n",
       "             ('dotings', 26),\n",
       "             ('egyptians', 26),\n",
       "             ('ibis', 26),\n",
       "             ('roasted', 26),\n",
       "             ('river', 26),\n",
       "             ('horse', 52),\n",
       "             ('mummies', 26),\n",
       "             ('creatures', 26),\n",
       "             ('huge', 52),\n",
       "             ('bake', 26),\n",
       "             ('houses', 52),\n",
       "             ('pyramids', 26),\n",
       "             ('simple', 26),\n",
       "             ('sailor', 156),\n",
       "             ('mast', 78),\n",
       "             ('plumb', 26),\n",
       "             ('forecastle', 52),\n",
       "             ('royal', 26),\n",
       "             ('true', 104),\n",
       "             ('rather', 208),\n",
       "             ('order', 130),\n",
       "             ('make', 260),\n",
       "             ('jump', 52),\n",
       "             ('spar', 52),\n",
       "             ('grasshopper', 26),\n",
       "             ('may', 312),\n",
       "             ('thing', 104),\n",
       "             ('unpleasant', 26),\n",
       "             ('enough', 338),\n",
       "             ('touches', 26),\n",
       "             ('sense', 78),\n",
       "             ('honour', 26),\n",
       "             ('particularly', 52),\n",
       "             ('established', 26),\n",
       "             ('family', 26),\n",
       "             ('van', 26),\n",
       "             ('rensselaers', 26),\n",
       "             ('randolphs', 26),\n",
       "             ('hardicanutes', 26),\n",
       "             ('putting', 52),\n",
       "             ('tar', 52),\n",
       "             ('pot', 26),\n",
       "             ('been', 468),\n",
       "             ('lording', 26),\n",
       "             ('schoolmaster', 52),\n",
       "             ('making', 130),\n",
       "             ('tallest', 26),\n",
       "             ('boys', 52),\n",
       "             ('awe', 26),\n",
       "             ('transition', 52),\n",
       "             ('keen', 26),\n",
       "             ('assure', 26),\n",
       "             ('decoction', 26),\n",
       "             ('seneca', 26),\n",
       "             ('stoics', 26),\n",
       "             ('enable', 26),\n",
       "             ('grin', 52),\n",
       "             ('bear', 52),\n",
       "             ('even', 130),\n",
       "             ('wears', 26),\n",
       "             ('hunks', 52),\n",
       "             ('orders', 26),\n",
       "             ('broom', 26),\n",
       "             ('sweep', 52),\n",
       "             ('decks', 26),\n",
       "             ('indignity', 26),\n",
       "             ('amount', 26),\n",
       "             ('weighed', 52),\n",
       "             ('scales', 26),\n",
       "             ('new', 286),\n",
       "             ('testament', 26),\n",
       "             ('think', 182),\n",
       "             ('archangel', 26),\n",
       "             ('gabriel', 26),\n",
       "             ('thinks', 182),\n",
       "             ('anything', 52),\n",
       "             ('less', 52),\n",
       "             ('promptly', 26),\n",
       "             ('obey', 26),\n",
       "             ('instance', 26),\n",
       "             ('ai', 104),\n",
       "             (\"n't\", 624),\n",
       "             ('slave', 26),\n",
       "             ('well', 208),\n",
       "             ('however', 208),\n",
       "             ('captains', 26),\n",
       "             ('thump', 52),\n",
       "             ('punch', 26),\n",
       "             ('satisfaction', 26),\n",
       "             ('knowing', 78),\n",
       "             ('everybody', 26),\n",
       "             ('else', 208),\n",
       "             ('served', 26),\n",
       "             ('either', 78),\n",
       "             ('physical', 26),\n",
       "             ('point', 52),\n",
       "             ('view', 52),\n",
       "             ('so', 1066),\n",
       "             ('universal', 26),\n",
       "             ('passed', 78),\n",
       "             ('hands', 78),\n",
       "             ('rub', 26),\n",
       "             ('shoulder', 26),\n",
       "             ('blades', 26),\n",
       "             ('again', 286),\n",
       "             ('always', 104),\n",
       "             ('paying', 78),\n",
       "             ('trouble', 26),\n",
       "             ('whereas', 26),\n",
       "             ('pay', 78),\n",
       "             ('single', 52),\n",
       "             ('penny', 78),\n",
       "             ('heard', 208),\n",
       "             ('contrary', 26),\n",
       "             ('difference', 52),\n",
       "             ('between', 234),\n",
       "             ('paid', 26),\n",
       "             ('act', 52),\n",
       "             ('perhaps', 130),\n",
       "             ('uncomfortable', 52),\n",
       "             ('infliction', 26),\n",
       "             ('orchard', 26),\n",
       "             ('thieves', 26),\n",
       "             ('entailed', 26),\n",
       "             ('us', 104),\n",
       "             ('paid,--what', 26),\n",
       "             ('compare', 52),\n",
       "             ('urbane', 26),\n",
       "             ('activity', 26),\n",
       "             ('receives', 26),\n",
       "             ('really', 104),\n",
       "             ('marvellous', 104),\n",
       "             ('considering', 26),\n",
       "             ('earnestly', 26),\n",
       "             ('believe', 26),\n",
       "             ('root', 26),\n",
       "             ('earthly', 52),\n",
       "             ('ills', 26),\n",
       "             ('monied', 26),\n",
       "             ('enter', 52),\n",
       "             ('heaven', 104),\n",
       "             ('ah', 26),\n",
       "             ('cheerfully', 26),\n",
       "             ('consign', 26),\n",
       "             ('perdition', 26),\n",
       "             ('finally', 26),\n",
       "             ('wholesome', 26),\n",
       "             ('exercise', 26),\n",
       "             ('pure', 26),\n",
       "             ('air', 104),\n",
       "             ('fore', 26),\n",
       "             ('castle', 26),\n",
       "             ('deck', 52),\n",
       "             ('far', 104),\n",
       "             ('prevalent', 26),\n",
       "             ('astern', 26),\n",
       "             ('violate', 26),\n",
       "             ('pythagorean', 26),\n",
       "             ('maxim', 26),\n",
       "             ('quarter', 52),\n",
       "             ('gets', 26),\n",
       "             ('atmosphere', 26),\n",
       "             ('second', 104),\n",
       "             ('sailors', 78),\n",
       "             ('breathes', 26),\n",
       "             ('commonalty', 26),\n",
       "             ('leaders', 52),\n",
       "             ('many', 104),\n",
       "             ('things', 130),\n",
       "             ('suspect', 26),\n",
       "             ('wherefore', 26),\n",
       "             ('after', 234),\n",
       "             ('repeatedly', 26),\n",
       "             ('smelt', 52),\n",
       "             ('merchant', 26),\n",
       "             ('whaling', 234),\n",
       "             ('invisible', 26),\n",
       "             ('police', 26),\n",
       "             ('fates', 52),\n",
       "             ('has', 104),\n",
       "             ('constant', 26),\n",
       "             ('surveillance', 26),\n",
       "             ('secretly', 26),\n",
       "             ('dogs', 26),\n",
       "             ('influences', 26),\n",
       "             ('unaccountable', 104),\n",
       "             ('answer', 130),\n",
       "             ('doubtless', 52),\n",
       "             ('formed', 52),\n",
       "             ('grand', 104),\n",
       "             ('programme', 26),\n",
       "             ('providence', 26),\n",
       "             ('drawn', 26),\n",
       "             ('came', 286),\n",
       "             ('brief', 26),\n",
       "             ('interlude', 26),\n",
       "             ('solo', 26),\n",
       "             ('extensive', 26),\n",
       "             ('performances', 26),\n",
       "             ('bill', 26),\n",
       "             ('run', 52),\n",
       "             ('contested', 26),\n",
       "             ('election', 26),\n",
       "             ('presidency', 26),\n",
       "             ('united', 26),\n",
       "             ('states', 26),\n",
       "             ('bloody', 26),\n",
       "             ('battle', 26),\n",
       "             ('affghanistan', 26),\n",
       "             ('exactly', 78),\n",
       "             ('stage', 52),\n",
       "             ('managers', 26),\n",
       "             ('put', 208),\n",
       "             ('shabby', 52),\n",
       "             ('others', 52),\n",
       "             ('magnificent', 26),\n",
       "             ('parts', 130),\n",
       "             ('tragedies', 26),\n",
       "             ('short', 78),\n",
       "             ('easy', 78),\n",
       "             ('genteel', 26),\n",
       "             ('comedies', 26),\n",
       "             ('jolly', 104),\n",
       "             ('farces', 26),\n",
       "             ('recall', 26),\n",
       "             ('circumstances', 52),\n",
       "             ('springs', 26),\n",
       "             ('motives', 52),\n",
       "             ('cunningly', 26),\n",
       "             ('presented', 26),\n",
       "             ('various', 52),\n",
       "             ('disguises', 26),\n",
       "             ('induced', 26),\n",
       "             ('performing', 26),\n",
       "             ('cajoling', 26),\n",
       "             ('delusion', 26),\n",
       "             ('choice', 26),\n",
       "             ('resulting', 26),\n",
       "             ('unbiased', 26),\n",
       "             ('freewill', 26),\n",
       "             ('discriminating', 26),\n",
       "             ('judgment', 26),\n",
       "             ('overwhelming', 26),\n",
       "             ('idea', 182),\n",
       "             ('whale', 260),\n",
       "             ('portentous', 78),\n",
       "             ('mysterious', 52),\n",
       "             ('monster', 26),\n",
       "             ('roused', 26),\n",
       "             ('curiosity', 52),\n",
       "             ('wild', 130),\n",
       "             ('seas', 156),\n",
       "             ('rolled', 156),\n",
       "             ('island', 78),\n",
       "             ('bulk', 26),\n",
       "             ('undeliverable', 26),\n",
       "             ('nameless', 78),\n",
       "             ('perils', 26),\n",
       "             ('attending', 26),\n",
       "             ('marvels', 26),\n",
       "             ('patagonian', 26),\n",
       "             ('sights', 26),\n",
       "             ('sounds', 78),\n",
       "             ('helped', 26),\n",
       "             ('sway', 26),\n",
       "             ('wish', 26),\n",
       "             ('inducements', 26),\n",
       "             ('tormented', 52),\n",
       "             ('everlasting', 52),\n",
       "             ('itch', 26),\n",
       "             ('remote', 26),\n",
       "             ('love', 26),\n",
       "             ('forbidden', 26),\n",
       "             ('barbarous', 26),\n",
       "             ('coasts', 26),\n",
       "             ('ignoring', 26),\n",
       "             ('good', 442),\n",
       "             ('quick', 26),\n",
       "             ('perceive', 26),\n",
       "             ('horror', 26),\n",
       "             ('social', 26),\n",
       "             ('since', 78),\n",
       "             ('friendly', 26),\n",
       "             ('terms', 26),\n",
       "             ('inmates', 26),\n",
       "             ('place', 390),\n",
       "             ('lodges', 26),\n",
       "             ('reason', 130),\n",
       "             ('welcome', 26),\n",
       "             ('flood', 26),\n",
       "             ('gates', 26),\n",
       "             ('wonder', 52),\n",
       "             ('swung', 26),\n",
       "             ('open', 104),\n",
       "             ('conceits', 26),\n",
       "             ('swayed', 26),\n",
       "             ('purpose', 52),\n",
       "             ('floated', 52),\n",
       "             ('inmost', 26),\n",
       "             ('endless', 26),\n",
       "             ('processions', 26),\n",
       "             ('mid', 26),\n",
       "             ('hooded', 26),\n",
       "             ('snow', 78),\n",
       "             ('stuffed', 52),\n",
       "             ('shirt', 104),\n",
       "             ('carpet', 26),\n",
       "             ('bag', 182),\n",
       "             ('tucked', 26),\n",
       "             ('arm', 286),\n",
       "             ('started', 26),\n",
       "             ('cape', 104),\n",
       "             ('horn', 52),\n",
       "             ('pacific', 26),\n",
       "             ('quitting', 26),\n",
       "             ('manhatto', 26),\n",
       "             ('duly', 26),\n",
       "             ('arrived', 52),\n",
       "             ('bedford', 104),\n",
       "             ('saturday', 78),\n",
       "             ('night', 624),\n",
       "             ('december', 26),\n",
       "             ('disappointed', 26),\n",
       "             ('learning', 26),\n",
       "             ('packet', 26),\n",
       "             ('nantucket', 182),\n",
       "             ('had', 858),\n",
       "             ('already', 26),\n",
       "             ('sailed', 26),\n",
       "             ('offer', 52),\n",
       "             ('till', 156),\n",
       "             ('following', 52),\n",
       "             ('monday', 26),\n",
       "             ('young', 130),\n",
       "             ('candidates', 26),\n",
       "             ('pains', 26),\n",
       "             ('penalties', 26),\n",
       "             ('stop', 208),\n",
       "             ('embark', 52),\n",
       "             ('related', 26),\n",
       "             ('doing', 26),\n",
       "             ('made', 338),\n",
       "             ('craft', 130),\n",
       "             ('fine', 104),\n",
       "             ('boisterous', 26),\n",
       "             ('everything', 26),\n",
       "             ('connected', 26),\n",
       "             ('famous', 26),\n",
       "             ('amazingly', 26),\n",
       "             ('pleased', 52),\n",
       "             ('late', 182),\n",
       "             ('gradually', 26),\n",
       "             ('monopolising', 26),\n",
       "             ('business', 130),\n",
       "             ('matter', 78),\n",
       "             ('behind', 26),\n",
       "             ('original', 52),\n",
       "             ('tyre', 26),\n",
       "             ('carthage;--the', 26),\n",
       "             ('dead', 130),\n",
       "             ('stranded', 52),\n",
       "             ('aboriginal', 26),\n",
       "             ('whalemen', 26),\n",
       "             ('red', 78),\n",
       "             ('sally', 26),\n",
       "             ('canoes', 26),\n",
       "             ('chase', 26),\n",
       "             ('leviathan', 52),\n",
       "             ('too', 364),\n",
       "             ('adventurous', 26),\n",
       "             ('sloop', 26),\n",
       "             ('forth', 26),\n",
       "             ('partly', 78),\n",
       "             ('laden', 26),\n",
       "             ('imported', 26),\n",
       "             ('cobblestones', 26),\n",
       "             ('throw', 26),\n",
       "             ('whales', 52),\n",
       "             ('discover', 26),\n",
       "             ('risk', 26),\n",
       "             ('harpoon', 135),\n",
       "             ('bowsprit', 26),\n",
       "             ('day', 156),\n",
       "             ('another', 130),\n",
       "             ('ere', 78),\n",
       "             ('destined', 26),\n",
       "             ('port', 26),\n",
       "             ('became', 78),\n",
       "             ('concernment', 26),\n",
       "             ('eat', 26),\n",
       "             ('meanwhile', 78),\n",
       "             ('dubious', 26),\n",
       "             ('nay', 52),\n",
       "             ('dark', 208),\n",
       "             ('dismal', 52),\n",
       "             ('bitingly', 26),\n",
       "             ('cold', 182),\n",
       "             ('cheerless', 26),\n",
       "             ('anxious', 26),\n",
       "             ('grapnels', 26),\n",
       "             ('sounded', 26),\n",
       "             ('pocket', 78),\n",
       "             ('only', 364),\n",
       "             ('brought', 26),\n",
       "             ('pieces', 26),\n",
       "             ('silver,--so', 26),\n",
       "             ('wherever', 52),\n",
       "             ('said', 494),\n",
       "             ('stood', 260),\n",
       "             ('middle', 130),\n",
       "             ('dreary', 52),\n",
       "             ('shouldering', 26),\n",
       "             ('comparing', 26),\n",
       "             ('gloom', 26),\n",
       "             ('darkness', 78),\n",
       "             ('wisdom', 26),\n",
       "             ('conclude', 26),\n",
       "             ('lodge', 26),\n",
       "             ('dear', 26),\n",
       "             ('sure', 130),\n",
       "             ('inquire', 26),\n",
       "             ('price', 26),\n",
       "             ('halting', 26),\n",
       "             ('steps', 26),\n",
       "             ('paced', 26),\n",
       "             ('sign', 156),\n",
       "             ('crossed', 52),\n",
       "             ('harpoons\"--but', 26),\n",
       "             ('looked', 156),\n",
       "             ('expensive', 52),\n",
       "             ('further', 104),\n",
       "             ('bright', 52),\n",
       "             ('windows', 52),\n",
       "             ('fish', 78),\n",
       "             ('inn', 78),\n",
       "             ('fervent', 26),\n",
       "             ('rays', 26),\n",
       "             ('seemed', 416),\n",
       "             ('melted', 26),\n",
       "             ('packed', 52),\n",
       "             ('ice', 104),\n",
       "             ('house', 286),\n",
       "             ('everywhere', 26),\n",
       "             ('congealed', 26),\n",
       "             ('frost', 104),\n",
       "             ('lay', 234),\n",
       "             ('inches', 52),\n",
       "             ('thick', 52),\n",
       "             ...])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.word_counts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "vocabulary_size = len(tokenizer.word_counts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2717"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocabulary_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(sequences)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "sequences = np.array(sequences)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 956,   14,  263, ..., 2712,   14,   24],\n",
       "       [  14,  263,   51, ...,   14,   24,  957],\n",
       "       [ 263,   51,  261, ...,   24,  957,    5],\n",
       "       ...,\n",
       "       [ 952,   12,  166, ...,  262,   53,    2],\n",
       "       [  12,  166, 2711, ...,   53,    2, 2717],\n",
       "       [ 166, 2711,    3, ...,    2, 2717,   26]])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sequences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.utils import to_categorical"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = sequences[:,:-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = sequences[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = to_categorical(y, num_classes = vocabulary_size + 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq_len = X.shape[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(11312, 25)"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "import keras\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense,LSTM,Embedding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_model(vocabulary_size,seq_len):\n",
    "    model = Sequential()\n",
    "    model.add(Embedding(vocabulary_size, seq_len, input_length=seq_len))\n",
    "    model.add(LSTM(50, return_sequences=True))\n",
    "    model.add(LSTM(50))\n",
    "    model.add(Dense(50, activation='relu'))\n",
    "\n",
    "    model.add(Dense(vocabulary_size, activation='softmax'))\n",
    "    \n",
    "    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
    "   \n",
    "    model.summary()\n",
    "    \n",
    "    return model\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, 25, 25)            67950     \n",
      "_________________________________________________________________\n",
      "lstm_1 (LSTM)                (None, 25, 150)           105600    \n",
      "_________________________________________________________________\n",
      "lstm_2 (LSTM)                (None, 150)               180600    \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 150)               22650     \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 2718)              410418    \n",
      "=================================================================\n",
      "Total params: 787,218\n",
      "Trainable params: 787,218\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = create_model(vocabulary_size+1, seq_len)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pickle import dump,load"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/abishai/.local/lib/python3.7/site-packages/tensorflow_core/python/framework/indexed_slices.py:433: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.\n",
      "  \"Converting sparse IndexedSlices to a dense Tensor of unknown shape. \"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/2\n",
      "11312/11312 [==============================] - 10s 926us/step - loss: 6.8705 - accuracy: 0.0436\n",
      "Epoch 2/2\n",
      "11312/11312 [==============================] - 9s 788us/step - loss: 6.3877 - accuracy: 0.0529\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.callbacks.History at 0x7fb9e02c4d50>"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(X,y,batch_size = 128,epochs = 2,verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save('my_mobydick_model.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "dump(tokenizer,open('my_simpletokenizer','wb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.preprocessing.sequence import pad_sequences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_text(model, tokenizer, seq_len, seed_text, num_gen_words):\n",
    "    '''\n",
    "    INPUTS:\n",
    "    model : model that was trained on text data\n",
    "    tokenizer : tokenizer that was fit on text data\n",
    "    seq_len : length of training sequence\n",
    "    seed_text : raw string text to serve as the seed\n",
    "    num_gen_words : number of words to be generated by model\n",
    "    '''\n",
    "    \n",
    "    # Final Output\n",
    "    output_text = []\n",
    "    \n",
    "    # Intial Seed Sequence\n",
    "    input_text = seed_text\n",
    "    \n",
    "    # Create num_gen_words\n",
    "    for i in range(num_gen_words):\n",
    "        \n",
    "        # Take the input text string and encode it to a sequence\n",
    "        encoded_text = tokenizer.texts_to_sequences([input_text])[0]\n",
    "        \n",
    "        # Pad sequences to our trained rate (50 words in the video)\n",
    "        pad_encoded = pad_sequences([encoded_text], maxlen=seq_len, truncating='pre')\n",
    "        \n",
    "        # Predict Class Probabilities for each word\n",
    "        pred_word_ind = model.predict_classes(pad_encoded, verbose=0)[0]\n",
    "        \n",
    "        # Grab word\n",
    "        pred_word = tokenizer.index_word[pred_word_ind] \n",
    "        \n",
    "        # Update the sequence of input text (shifting one over with the new word)\n",
    "        input_text += ' ' + pred_word\n",
    "        \n",
    "        output_text.append(pred_word)\n",
    "        \n",
    "    # Make it look like a sentence.\n",
    "    return ' '.join(output_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['call',\n",
       " 'me',\n",
       " 'ishmael',\n",
       " 'some',\n",
       " 'years',\n",
       " 'ago',\n",
       " 'never',\n",
       " 'mind',\n",
       " 'how',\n",
       " 'long',\n",
       " 'precisely',\n",
       " 'having',\n",
       " 'little',\n",
       " 'or',\n",
       " 'no',\n",
       " 'money',\n",
       " 'in',\n",
       " 'my',\n",
       " 'purse',\n",
       " 'and',\n",
       " 'nothing',\n",
       " 'particular',\n",
       " 'to',\n",
       " 'interest',\n",
       " 'me',\n",
       " 'on']"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_sequences[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "random.seed(101)\n",
    "random_pick = random.randint(0,len(text_sequences))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "random_seed_text = text_sequences[random_pick]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['thought',\n",
       " 'i',\n",
       " 'to',\n",
       " 'myself',\n",
       " 'the',\n",
       " 'man',\n",
       " \"'s\",\n",
       " 'a',\n",
       " 'human',\n",
       " 'being',\n",
       " 'just',\n",
       " 'as',\n",
       " 'i',\n",
       " 'am',\n",
       " 'he',\n",
       " 'has',\n",
       " 'just',\n",
       " 'as',\n",
       " 'much',\n",
       " 'reason',\n",
       " 'to',\n",
       " 'fear',\n",
       " 'me',\n",
       " 'as',\n",
       " 'i',\n",
       " 'have']"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random_seed_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "seed_text = ' '.join(random_seed_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"thought i to myself the man 's a human being just as i am he has just as much reason to fear me as i have\""
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seed_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the'"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate_text(model,tokenizer,seq_len,seed_text=seed_text,num_gen_words=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}