Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ddev config file #111

Merged
merged 27 commits into from
Nov 8, 2023
Merged

Conversation

bmartinez287
Copy link
Collaborator

The Issue

This repo does not use a ddev file that requires one to install npm. I tried making my own a while back and ended up down a rabbit hole. It potentially adds a barrier to contrib.
Because ddev is already on my machine it would be nice to take advantage of it to among other things switch npm versions and have easy access to a database.

How This PR Solves The Issue

This PR adds the ddev config file. However, I ran into a few issues.

Network ddev_default created
Failed to get project(s): invalid app type: astro

If I set the project type to astro it will complain since it's not on the list and that makes sense I suppose but It made me think whether there should be a custom type or a generic type.

Additionally, I was trying to run ddev npm run dev and map the local HTTPs server port 4321 but when I tried it with web_extra_exposed_ports it did not work. It gave me a bad route error.

I might be missing some history on why some of these items are the way they are. I will look on Discord and see if I can find them.

@bmartinez287
Copy link
Collaborator Author

After looking at some plugins I wonder if Astro dev settings should be part of a plugin such as vite and others.
https://github.com/tyler36/ddev-viteserve
The vite one is a good example. That might still require a custom project type or some sort of tag for a project base.

@bmartinez287 bmartinez287 marked this pull request as draft October 9, 2023 04:57
Copy link
Member

@rfay rfay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - The next thing to add is to update the README explaining how to use it.

You could also get just slightly fancier and change the nginx config to use astro as its upstream instead of php-fpm, and then everything would be on ports 80 and 443.

.ddev/config.yaml Outdated Show resolved Hide resolved
.ddev/config.yaml Outdated Show resolved Hide resolved
.ddev/config.yaml Outdated Show resolved Hide resolved
.ddev/config.yaml Outdated Show resolved Hide resolved
.ddev/config.yaml Outdated Show resolved Hide resolved
Copy link
Member

@rfay rfay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems fine. If you wanted to automatlcally npm run dev you could do it with a web_extra_daemons stanza.

@bmartinez287
Copy link
Collaborator Author

I'm looking into the nginx config to switch from php-fpm but I'm not sure if I found the right settings there.

Additionally, Astro runs Vite underhood so I figure maybe the Vite ddev add-on or running vite as a web_extra_exposed_ports would do it but so far I haven't been able to get HMR Hot module reloading working with Astro Vite implementation. I saw a few support posts on discord about some getting it to work but they are using a standard vite install and I think Astro does a wrapper which might complicate it a bit.

The dev.config file works well as it is if one just runs npm run build but it be nice to solve it for npm run dev as well

@bmartinez287
Copy link
Collaborator Author

I get an astro.conf in /etc/supervisor/conf.d/astro.config after I do a ddev.ssh with the following settings.

[program:astro]
group=webextradaemons
command=bash -c "npm run dev || sleep 2"
directory=/var/www/html
autostart=false
autorestart=true
startretries=15
stdout_logfile=/var/tmp/logpipe
stdout_logfile_maxbytes=0
redirect_stderr=true

Which seems fine. Looking at this issue ddev/ddev#4872 but port 9998 or 9999 does not let me reach it it gives me a bad gateway.

I imagine this should be reachable http://ddev.com.ddev.site:9998/ and I thought that would give me the same output as npm run dev on a local environment. I must be missing a step since I got a bad gateway error.

@bmartinez287
Copy link
Collaborator Author

bmartinez287 commented Oct 13, 2023

time="2023-10-12T23:55:54Z" level=debug msg="'502 Bad Gateway' caused by: dial tcp 192.168.112.4:4321: connect: connection refused"
192.168.112.1 - - [12/Oct/2023:23:55:54 +0000] "GET /favicon.ico HTTP/2.0" 502 11 "-" "-" 51 "ddev-astrosite-web-4321-https@file" "http://ddev-ddev-astrosite-web:4321" 1ms

ddev logs --follow --time

@bmartinez287 bmartinez287 marked this pull request as ready for review October 13, 2023 20:47
@bmartinez287 bmartinez287 requested a review from rfay October 13, 2023 20:47
@bmartinez287
Copy link
Collaborator Author

Got it working, the issue was a missing -- that Atro Vite required to allow all hosts.
I added instructions to the README and updated the files.

Copy link
Member

@rfay rfay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Congrats on all the good work. Variety of suggestions, I haven't manually tested yet.

@@ -0,0 +1,35 @@
name: astrosite
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend just leaving this out if possible. It will then take the name of the directory it's in.

Suggested change
name: astrosite

# The extra -- is a Vite item
# https://github.com/vitejs/vite/discussions/3396
web_extra_daemons:
- name: astro-dev-deamon
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might as well spell it as expected.

Suggested change
- name: astro-dev-deamon
- name: astro-dev-daemon

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good

.ddev/config.yaml Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
@bmartinez287 bmartinez287 requested a review from rfay October 15, 2023 21:18
Copy link
Member

@rfay rfay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ddev npm run build gets

▶ src/pages/index.astro
node:internal/event_target:1033
  process.nextTick(() => { throw err; });

Since you're using web_extra_exposed_ports you shouldn't be telling people to hit http://127.0.0.1, should be named URL, for example https://ddev.com.ddev.site:4321 (which works)

But https://ddev.com.ddev.site:4321/blog gets me a 404.

The post-start hook shows By default it will be available at port 4321 - URL:4321, but there's no reason it can't show the full URL using ${DDEV_PRIMARY_URL}:4321

README.md Outdated
DDev already has all the dependencies included.

1. Run `ddev start && ddev npm install` set up the project’s dependencies.
3. `npm run build` can be found at the base URL and `npm run dev` is found at URL:4321. Dev has Vite HMR(hot module reloading) among other features. The site will automatically refresh as you work on it, displaying errors in the relevant console (terminal or browser).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
3. `npm run build` can be found at the base URL and `npm run dev` is found at URL:4321. Dev has Vite HMR(hot module reloading) among other features. The site will automatically refresh as you work on it, displaying errors in the relevant console (terminal or browser).
2. `ddev npm run build` can be found at the base URL and `ddev npm run dev` is found at URL:4321. Dev has Vite HMR(hot module reloading) among other features. The site will automatically refresh as you work on it, displaying errors in the relevant console (terminal or browser).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

${DDEV_PRIMARY_URL}:4321 - I was looking for that variable for a while haha, I will work on that.

http://127.0.0.1/ is for the local setup without DDev in case they just want to run npm but maybe I can make that clearer I will double check.

I did not see that build error I will check on that one as well.

Thanks for the feedback.

@bmartinez287
Copy link
Collaborator Author

I updated my OS to Sonoma, switched to Colima, and have been running into an odd issue where it gives me an operation not permitted.
Screenshot 2023-10-16 at 3 36 39 PM

Screenshot 2023-10-16 at 3 41 15 PM

I'm probably hitting an OS bug now 😑

That with ddev start && ddev npm cache clean --force && ddev npm install the fact one has to cache clean as sometimes it gets stuck in between.
More updates to follow.

@bmartinez287
Copy link
Collaborator Author

bmartinez287 commented Oct 16, 2023

Screenshot 2023-10-16 at 4 05 51 PM

DDev npm run build works well for the most part. But has that same error.

npm WARN tar TAR_ENTRY_ERROR EPERM: operation not permitted, open '/var/www/html/node_modules/@isaacs/cliui/node_modules/emoji-regex/package.json'
npm WARN tar TAR_ENTRY_ERROR EPERM: operation not permitted, open '/var/www/html/node_modules/timm/.github/workflows/run-tests.yml'
npm WARN tar TAR_ENTRY_ERROR EPERM: operation not permitted, open '/var/www/html/node_modules/tailwindcss/node_modules/postcss-selector-parser/dist/selectors/index.js'
npm WARN tar TAR_ENTRY_ERROR EPERM: operation not permitted, open '/var/www/html/node_modules/@jridgewell/trace-mapping/dist/types/sort.d.ts'
npm WARN tar TAR_ENTRY_ERROR EPERM: operation not permitted, open '/var/www/html/node_modules/pngjs/lib/packer.js'
npm WARN tar TAR_ENTRY_ERROR EPERM: operation not permitted, open '/var/www/html/node_modules/bottleneck/LICENSE'

@rfay
Copy link
Member

rfay commented Oct 16, 2023

Not sure what you were hitting there, especially if you had mutagen enabled. Can't tell much about what you were running against with Colima without knowing what Colima config you were using and whether you had mutagen enabled. However, note that Lima just had an update...

@bmartinez287
Copy link
Collaborator Author

bmartinez287 commented Oct 16, 2023

https://ddev.readthedocs.io/en/stable/users/install/docker-installation/#macos
I'm running the docs install but more specifically.
colima start --cpu 4 --memory 6 --disk 100 --vm-type=qemu --mount-type=sshfs --dns=1.1.1.1
I'm doing brew docker version 24.0.6 and brew colima version 0.5.6.

Yeah, I might have gone too close to the bleeding edge now. O well, it will work eventually hahaha.

@rfay
Copy link
Member

rfay commented Oct 16, 2023

Not sure how you can get permissions errors with that setup if using mutagen.

New limactl version (which makes a big difference) is

limactl version 0.18.0

@bmartinez287
Copy link
Collaborator Author

Ups, I was not using mutagen. Thanks for adding that note.

@bmartinez287
Copy link
Collaborator Author

bmartinez287 commented Oct 16, 2023

It's working!!! and I think is ready.

To run it in ddev make sure to run ddev start && ddev npm install so the daemon won't crash the web container.

If it fails then run ddev npm cache clean --force && ddev npm install && ddev restart

Sometimes the npm cache will get stuck if one compiles it locally and then inside the container. Doing both creates chaos and requires a little more knowledge of the npm cache.

@bmartinez287 bmartinez287 requested a review from rfay October 16, 2023 21:02
README.md Outdated
@@ -65,7 +65,7 @@ All commands are run from the root of the project, from a terminal:

DDev already has all the dependencies included.

1. Run `ddev start && ddev npm install` set up the project’s dependencies.
1. Run `ddev start && ddev npm cache clean --force && ddev npm install ` set up the project’s dependencies.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Safe to get rid of the extra space unless that was important:

Suggested change
1. Run `ddev start && ddev npm cache clean --force && ddev npm install ` set up the project’s dependencies.
1. Run `ddev start && ddev npm cache clean --force && ddev npm install` set up the project’s dependencies.

README.md Outdated
@@ -65,7 +65,7 @@ All commands are run from the root of the project, from a terminal:

DDev already has all the dependencies included.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure where “DDev” came from, but it should always be referred to as DDEV the product or ddev the command.

Suggested change
DDev already has all the dependencies included.
DDEV already has all the dependencies included.

README.md Outdated
@@ -65,7 +65,7 @@ All commands are run from the root of the project, from a terminal:

DDev already has all the dependencies included.

1. Run `ddev start && ddev npm install` set up the project’s dependencies.
1. Run `ddev start && ddev npm cache clean --force && ddev npm install ` set up the project’s dependencies.
3. `ddev npm run build` can be found at the base URL and `ddev npm run dev` is found at https://<projectname>.ddev.site:4321. Dev has Vite HMR(hot module reloading) among other features. The site will automatically refresh as you work on it, displaying errors in the relevant console (terminal or browser).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 → 2, fix spacing, reword slightly for clarity:

Suggested change
3. `ddev npm run build` can be found at the base URL and `ddev npm run dev` is found at https://<projectname>.ddev.site:4321. Dev has Vite HMR(hot module reloading) among other features. The site will automatically refresh as you work on it, displaying errors in the relevant console (terminal or browser).
2. `ddev npm run build` can be found at the base URL and `ddev npm run dev` is found at https://<projectname>.ddev.site:4321. The dev server has Vite HMR (hot module reloading) among other features. The site will automatically refresh as you work on it, displaying errors in the relevant terminal or browser console.

What do you mean by ddev npm run build can be found at the base URL?

I know that running the build command generates a static version of the site into the dist/ directory, which you can run the preview command to serve, but the wording of this seems sort of ambiguous. Similarly with “ddev npm run dev is found at”—the command isn’t found at a browser address, but running it starts a server you can access from the browser at that address. I know you’re aware of this, but the wording leaves it unclear for someone figuring it out.

Copy link
Collaborator Author

@bmartinez287 bmartinez287 Oct 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I can see why it could be confusing. I updated the text to say.

ddev npm run build can be found at https://.ddev.site and ddev npm run dev is found at https://.ddev.site:4321.The dev server has Vite HMR (hot module reloading) among other features. The site will automatically refresh as you work on it, displaying errors in the relevant terminal or browser console.

README.md Outdated

To generate a static copy of the site, run `ddev npm run build`. The contents of the `dist/` folder are exactly what get [deployed to Cloudflare Pages](#build--deployment). You can preview locally by running `ddev npm run preview` or using a tool like [`serve`](https://www.npmjs.com/package/serve).

#### Without DDev setup
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#### Without DDev setup
#### Setup Without DDEV

README.md Outdated
6. Run `npm run dev` to start Astro’s dev server.
7. Visit the URL displayed in your terminal. (Probably `http://127.0.0.1:3000/`.) The site will automatically refresh as you work on it, displaying errors in the relevant console (terminal or browser).
3. Run `npm run dev` to start Astro’s dev server. If it fails then run `npm cache clean --force && npm install && npm run dev`.
4. Visit the URL displayed in your terminal. (Probably `http://localhost:4321/`.) The site will automatically refresh as you work on it, displaying errors in the relevant console (terminal or browser).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
4. Visit the URL displayed in your terminal. (Probably `http://localhost:4321/`.) The site will automatically refresh as you work on it, displaying errors in the relevant console (terminal or browser).
4. Visit the URL displayed in your terminal. (Probably `http://localhost:4321/`.) The site will automatically refresh as you work on it, displaying errors in the relevant terminal or browser console.

README.md Outdated

To generate a static copy of the site, run `npm run build`. The contents of the `dist/` folder are exactly what get [deployed to Cloudflare Pages](#build--deployment). You can preview locally by running `npm run preview` or using a tool like [`serve`](https://www.npmjs.com/package/serve).


#### Switching from without Ddev to with DDev local environment.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use correct “DDEV”s and shorten:

Suggested change
#### Switching from without Ddev to with DDev local environment.
#### Switching from Without DDEV to With DDEV

README.md Outdated

#### Switching from without Ddev to with DDev local environment.

Make sure to delete your node vendor folder everytime you switch and run npm install. The change in architecture can create odd issues otherwise.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Be specific, use backticks for command:

Suggested change
Make sure to delete your node vendor folder everytime you switch and run npm install. The change in architecture can create odd issues otherwise.
Make sure to delete your `node_modules/` directory and run `npm install`. The change in architecture can create odd issues otherwise.

@bmartinez287
Copy link
Collaborator Author

bmartinez287 commented Oct 19, 2023

I'm running into a few odd issues here among them.
ddev start && ddev npm install && ddev npm run build might be too long of an operation. It seems to get killed before it completes execution.

ddev start && ddev npm install works but for first-time setups, they would need the build folder to since that's where https://.ddev.site takes you.

Additionally, web_extra_daemons sometimes runs before ddev npm install and that breaks the build as it won't have anything to build upon. command: "sleep 3 && npm run dev -- --host" I added the sleep 3 to help with it since sometimes it is a race condition that could be mitigated by it. Is there another way to make ddev npm install run before web_extra_daemons?

The post-start hook would not do it because that has the same race condition issue.

@rfay
Copy link
Member

rfay commented Oct 19, 2023

If you're seeing killed check your Docker memory allocation.

@bmartinez287
Copy link
Collaborator Author

This PR works but the daemon running before npm install has completed does concern me a bit. It fails if it can't find those files. I feel like I need a hook between pre-start and post-start so I can still do the npm install but it won't run the daemon until is over with it.

@rfay
Copy link
Member

rfay commented Oct 27, 2023

I'll try to take a look at this before long.

@rfay
Copy link
Member

rfay commented Nov 8, 2023

Sorry I lost track of this.

@bmartinez287
Copy link
Collaborator Author

No worries, im sure you been pinned down after traveling fora while.

@rfay rfay force-pushed the 20231009_ddev_config_file branch from 9a0e5ab to a002105 Compare November 8, 2023 14:06
@rfay
Copy link
Member

rfay commented Nov 8, 2023

Rebased

@rfay
Copy link
Member

rfay commented Nov 8, 2023

I'm seeing this kind of behavior:

> @ddev/[email protected] dev
> astro dev --host

  🚀  astro  v3.4.4 started in 554ms

  ┃ Local    http://localhost:4321/
  ┃ Network  http://172.27.0.2:4321/
             http://172.26.0.4:4321/

04:48:20 PM [content] Watching src/content/ for changes
04:48:21 PM [content] Types generated
04:48:21 PM [astro] update /.astro/types.d.ts
2023-11-08 16:48:22,015 INFO waiting for child_exit_monitor, php-fpm, nginx, mailpit, astro-dev-daemon to die
2023-11-08 16:48:25,023 INFO waiting for child_exit_monitor, php-fpm, nginx, mailpit, astro-dev-daemon to die
2023-11-08 16:48:26,030 WARN killing 'astro-dev-daemon' (2493) with SIGKILL
2023-11-08 16:48:27,031 INFO stopped: astro-dev-daemon (terminated by SIGKILL)
2023-11-08 16:48:27,033 INFO stopped: mailpit (exit status 0)
2023-11-08 16:48:27,068 INFO stopped: nginx (exit status 0)
[08-Nov-2023 16:48:27] NOTICE: Terminating ...
[08-Nov-2023 16:48:27] NOTICE: exiting, bye-bye!
2023-11-08 16:48:27,079 INFO stopped: php-fpm (exit status 0)
2023-11-08 16:48:27,079 INFO stopped: child_exit_monitor (terminated by SIGTERM)

Anyway, it comes up but I think astro dies soon, and then the whole web container gets brought down. I've seen this lately running npm run dev on the host a few times as well, it starts up and then dies.

Could you take a look and see what you see?

@rfay
Copy link
Member

rfay commented Nov 8, 2023

Never mind. I didn't follow the instructions.

Copy link
Member

@rfay rfay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working great for me, thanks! Minor suggestion in README.

webserver_type: nginx-fpm
additional_hostnames: []
additional_fqdns: []
performance_mode: mutagen # Important to avoid EPERM errors
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd love to know what mode you were using and what docker provider you were using that gave you trouble.

README.md Outdated Show resolved Hide resolved
Co-authored-by: Randy Fay <[email protected]>
@rfay rfay merged commit a588d01 into ddev:main Nov 8, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants