Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache book results from the NYT best sellers to the database #30

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

YOURETHEBEST-AROUND
Copy link
Contributor

Completed feature 62.

Currently, I've implemented it so that the server pulls weekly lists from NY Times back to 2013/01/01 and stores the all books found in the database with Title, Author, List Date, ISBN13 values (others can easily be added). This should make it possible to query the database for bestsellers from a month/year?
setInterval function is used to make requests approx. every 300 milliseconds which seems to avoid call limit issues.
Currently implemented as a route in route.js so that visiting /populateNY on the server will cause it to pull and populate the database. (The function wipes the book collection first (so duplicates aren't created) and I assume calls the NY api a lot so I recommend not doing this too much unless you don't mind not being able to pull books for a day or so).
I wasn't sure about putting the function here but wasn't sure where else to put it or how to have the server run it. If there's a better way I'm hoping it won't be much more difficult than copy pasting this code elsewhere.

@ghost
Copy link

ghost commented Apr 18, 2018

Didn't work for me, the app crashed after running npm install; npm run dev

blindcant@maynard api $ npm run dev

[email protected] dev /home/blindcant/dev/web/nookr.io/api
nodemon --watch /src ./src/app.js

[nodemon] 1.17.3
[nodemon] to restart at any time, enter rs
[nodemon] watching: /src
[nodemon] starting node ./src/app.js
internal/modules/cjs/loader.js:550
throw err;
^

Error: Cannot find module 'axios'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:548:15)
at Function.Module._load (internal/modules/cjs/loader.js:475:25)
at Module.require (internal/modules/cjs/loader.js:598:17)
at require (internal/modules/cjs/helpers.js:11:18)
at Object. (/home/blindcant/dev/web/nookr.io/api/src/routes.js:5:15)
at Module._compile (internal/modules/cjs/loader.js:654:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:665:10)
at Module.load (internal/modules/cjs/loader.js:566:32)
at tryModuleLoad (internal/modules/cjs/loader.js:506:12)
at Function.Module._load (internal/modules/cjs/loader.js:498:3)
at Module.require (internal/modules/cjs/loader.js:598:17)
at require (internal/modules/cjs/helpers.js:11:18)
at Object. (/home/blindcant/dev/web/nookr.io/api/src/app.js:40:1)
at Module._compile (internal/modules/cjs/loader.js:654:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:665:10)
at Module.load (internal/modules/cjs/loader.js:566:32)
[nodemon] app crashed - waiting for file changes before starting...

Need npm install axios and then npm install moment

@ghost
Copy link

ghost commented Apr 18, 2018

Working fine after the correct packages being installed. The mongoDB is being populated and I can see that in robo 3t. Need to connect to localhost:27017 to connect to the mongoDB. Just waiting for the list to finish before squash and merge.

@ghost
Copy link

ghost commented Apr 18, 2018

API error message: 'API rate limit exceeded'

@ghost
Copy link

ghost commented Apr 18, 2018

Going to block for now so it can be redesigned as discussed with Luke.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant