Add automatic scraping #17
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Very basic scraper for practice
This one reads a bunch of inspirational quotes and creates a table to be saved into a .csv file
* Move documentation into its own directory * Update documentation, fixes #8. * Fix misaligned table for the section properties * Reorganize scraper repository (#12) * Delete old scrapers * Move documentation into a separate directory * Remove old json files from gitignore * Add some basic goals to the README * Basic start of Clojure course catalog scraper * Clean up Clojure, it was a fun experiment while it lasted * Merge sis.rpi.edu/course_scrapper.py into lily-rewrite * Rename course scraper script * Delete data/ directory * Create python-app.yml Testing GitHub Actions with auto-executing Python files as soon as they're published to the repository Co-authored-by: Lily Ritter <[email protected]> Co-authored-by: Trevor Brunette <[email protected]> Co-authored-by: Lily Ritter <[email protected]>
Should have fixed the error with not scraping in the wrong place, also added working directory in order to upload to the data repository
Trying again with pushing to data repository
|
So we currently have a lil problem that running this action will accidentally push the Thanks for the contributions!
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
python-app.yml will run every Python file in the scraper directory and push any files that said Python files create to the scraper directory.