Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: auto add dem file if not provided by the user #380

Open
wants to merge 15 commits into
base: develop
Choose a base branch
from

Conversation

aryalsaurav
Copy link
Collaborator

@aryalsaurav aryalsaurav commented Dec 4, 2024

What type of PR is this? (check all applicable)

  • πŸ• Feature
  • πŸ› Bug Fix
  • πŸ“ Documentation
  • πŸ§‘β€πŸ’» Refactor
  • βœ… Test
  • πŸ€– Build or CI
  • ❓ Other (please specify)

Describe this PR

This PR introduces a feature where if the Project creator wants DEM file to be automatically uploaded instead of him uploading the file. Scrapy Bot will crawl the "Jaxa" website for the dem file and if there are multiple images that falls in that selected range then it merges all into one DEM file and later uploaded to S3 bucket.

Review Guide

Notes for the reviewer. How to test this change?

Checklist before requesting a review

  • πŸ“– Read the HOT Code of Conduct: https://docs.hotosm.org/code-of-conduct
  • πŸ‘·β€β™€οΈ Create small PRs. In most cases, this will be possible.
  • βœ… Provide tests for your changes.
  • πŸ“ Use descriptive commit messages.
  • πŸ“— Update any related documentation and include any relevant screenshots.

@aryalsaurav aryalsaurav requested a review from Pradip-p December 4, 2024 11:52
@github-actions github-actions bot added enhancement New feature or request backend Related to backend code labels Dec 4, 2024
@aryalsaurav aryalsaurav requested a review from nrjadkry December 4, 2024 11:52
@@ -0,0 +1,92 @@
# Scrapy settings for jaxa project
#
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, you set up a Scrapy app(Jaxa), which is great! However, I highly recommend using Scrapy's Core API. It simplifies the workflow and is perfect for single-page scripts without requiring a full project setup.

Here’s an example:

from scrapy.crawler import CrawlerRunner
from scrapy import Spider
from twisted.internet import reactor

class ExampleSpider(Spider):
    name = "example"
    start_urls = ["https://example.com"]

    def parse(self, response):
        print("Scraped content:", response.body[:100])

runner = CrawlerRunner()
runner.crawl(ExampleSpider)
runner.join().addBoth(lambda _: reactor.stop())
reactor.run()

@nrjadkry nrjadkry requested a review from spwoodcock December 5, 2024 03:48
@aryalsaurav aryalsaurav force-pushed the feat/auto-add-dem-file branch 2 times, most recently from 6b7ef4c to cae4a94 Compare December 5, 2024 04:37
@aryalsaurav aryalsaurav force-pushed the feat/auto-add-dem-file branch from 304d16a to 1f0baf5 Compare December 5, 2024 05:46
@aryalsaurav aryalsaurav force-pushed the feat/auto-add-dem-file branch from 1fa5675 to 850f07a Compare December 5, 2024 10:38
@bijayrauniyar0
Copy link
Collaborator

  • Add radio buttons to ask user if they want to upload DEM or they need from server

Screenshots

image

@spwoodcock
Copy link
Member

  • Add radio buttons to ask user if they want to upload DEM or they need from server

Screenshots

image

Nice πŸ‘

Is the text supposed to say 'approval for lock task'?

Also having it marked mandatory doesn't make sense if you provide a default and its a radio.

Perhaps it might help adding a small bit of explanation:

'We provide a global elevation model that should be suitable for most cases, but you may wish to upload a more detailed custom model in certain scenarios (for example steep mountainous terrain)'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Related to backend code enhancement New feature or request frontend
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants