Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for suggest2 API and modernization #107

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

tfmorris
Copy link

This is mainly an information PR for some work in progress to see if there's any interest. I added support for the suggest2 API which does keyword searching of all variants in addition to left anchored searches, but it wasn't as big an improvement as I hoped. Things could also be extended to support the many new information types that the LoC now includes (e.g. BIBFRAME works).

In addition to the new API, I added a bunch of additional stuff including:

  • Added support for the front end specified limit parameter to control number of choices returned
  • Added HTTP header identifying service as requested by LoC
  • Add HTTP timeouts, retries, and performance metrics (as well as switching to HTTPS)
  • switched to RapidFuzz instead of Fuzzy Wuzzy because of licensing chaos (plus it's faster)
  • switch to Conda for dependencies, updated all dependencies, and dropped unused ones
  • refactored code to reduce some of the redundancy
  • dropped Python 2 support

I couldn't find any documentation on the didyoumean API and it doesn't seem to return many results, so I'm not sure how useful it is. More generally it's unclear to me what the best API or APIs to use is and the Library of Congress doesn't really provide much guidance. It would probably take more time than I have available currently to do enough experimentation to figure it out, but I'd be happy to accept feedback from an expert.

tfmorris and others added 6 commits October 27, 2022 21:42
* Add support for suggest2 API (disabled by default)
* Honor limit parameter for number of choices returned
* Add HTTP timeouts & retries with backoff
* Add HTTP metrics
* Add HTTP request header as required by LoC
* refactor to minimize redundant code
Turn on caching by default
- Cherry-picked from yorkulibraries/lc-reconcile by @ruebot
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants