Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

best way to disable exiftool2 #37

Open
svanzoest opened this issue May 29, 2017 · 3 comments
Open

best way to disable exiftool2 #37

svanzoest opened this issue May 29, 2017 · 3 comments

Comments

@svanzoest
Copy link

svanzoest commented May 29, 2017

I am running scrappy in a serverless environment where I have limited ability to spawn processes and definitely do not have access to perl. What is the best way to disable exiftool2 spawn? I attempted to only use the html plugin with scrapeUrl, but I either don't have the syntax right, or it still manages to call exiftool2.

I appreciate any insight you might be able to provide.

@svanzoest
Copy link
Author

Turns out this is caused by the scrappy.helpers.iconSelector extract helper that I was using later.
I'd like to have access to the icons from the site.

@blakeembrey
Copy link
Member

Two thoughts:

  1. Let's improve https://github.com/blakeembrey/node-scrappy/blob/d63aaa5613901594730105a895efb397597d0d6d/src/extract/helpers/icon-selector.ts#L32-L33 so it uses itself (instead of using the default configurations), that way you can disable https://github.com/blakeembrey/node-scrappy/blob/d63aaa5613901594730105a895efb397597d0d6d/src/scrape/plugins/exif-data.ts which is your issue (this is used for all scrapers)
  2. Let's figure out if it's also possible to bundle the perl file into a single executable so it can run on lambda

@svanzoest
Copy link
Author

svanzoest commented May 29, 2017

That sounds good to me. Ideally not leaving javascript at all would help.

3.) Similarly digitalbazaar/jsonld.js#184 was giving me some grief, as I am using webpack to build the serverless function, so there no longer is a package.json.

I'd love to help to at least get 1. going and then follow up with 2. I am not terribly familiar with typescript, but quite fluent in javascript, exif and media extraction in general.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants