Git support

[mindplay-dk]

Neat project! 😃

How about adding support for git commands?

• https://www.npmjs.com/package/isomorphic-git

• https://www.npmjs.com/package/g4c

This would be very intuitive - I'm always surprised when I instinctively type a git command on StackBlitz, or attempt to npm install from a git: source, and it doesn't work. 😅

(Note that I couldn't immediately get g4c to actually work on StackBlitz - the author says it should work, which presumably means it would work on WebContainers, but no luck yet. Very new project though. I opened an issue and asked.)

What else do you have planned? Just playing around for now, or do you have a loose roadmap in mind?

[FossPrime]

g4c

Fixed in 1.1.0 ... caused by a feature that was reading from package.json's git

https://stackblitz.com/edit/g4c-demo

[mindplay-dk]

@FossPrime Thanks, I can see it working in that main.sh script, but it still doesn't work from the command-line afterwards. Also, it looks like the git binary is now missing? (I don't know turbo or pnpm so it's all a bit mysterious to me.)

I thought npm worked in StackBlitz - looks like this is actually a shim passing to turbo somehow? They had to build their own package manager to get this to work with WebContainers?

I guess we're still some years from something like this being an options. 😅

[FossPrime]

@FossPrime Thanks, I can see it working in that main.sh script, but it still doesn't work from the command-line afterwards. Also, it looks like the git binary is now missing? (I don't know turbo or pnpm so it's all a bit mysterious to me.)

I thought npm worked in StackBlitz - looks like this is actually a shim passing to turbo somehow? They had to build their own package manager to get this to work with WebContainers?

I guess we're still some years from something like this being an options. 😅

It clones to the current directory... so if there's a conflict... like package.json it safely exits.

I'll make 1.2 create a directory and cd into it before cloning...

The whole thing was made mostly as a way to get Gitlab support on Codesandbox... I just wrote it with javascript because it's what I knew... oh and I hate bash scripts. 😅

[FossPrime]

1.2.0 is out and it works on VSLite... but the React Tree View is caput...

I have two options for bringing back the git alias...

Always hijack like before, but make sure PATH's don't cause infinite loops
OR only hijack if I detect you're missing git... which is tricky to do with YARN 2.

This is a lot easier to do if I only have to detect stackblitz... but CloudFlow has an okay version of this too... Replit has a surprisingly similarly problematic version too.

[TheUltDev]

@mindplay-dk I updated the ROADMAP. There's a lot more to flesh out, most is in my head, but that's the gist.

@FossPrime awesome work! I think I can add the alias in the terminal input, before it reaches the shell. nvm that won't work.

As far as the file tree goes, yup that's a known issue, it doesn't update subfolders well. It needs to be fully refactored, both the scanning of the directory and the updating of the tree.

I will keep this issue open until we have a good UX for git.

[TheUltDev]

Update: I pushed a change to run alias git='npx g4c' when the jsh process starts.

Now you can run: git clone 'https://github.com/kat-tax/vslite.git' vslite directly.

Only thing left is it would be nice to automatically install g4c globally without needing npx. Haven't figured out how to do that in WebContainers yet, not sure if you can.

[FossPrime]

Update: I pushed a change to run alias git='npx g4c' when the jsh process starts.

Now you can run: git clone 'https://github.com/kat-tax/vslite.git' vslite directly.

Only thing left is it would be nice to automatically install g4c globally without needing npx. Haven't figured out how to do that in WebContainers yet, not sure if you can.

It's pretty fast as is... For global installs I've found pnpm to be better.

See the funky package.json in the demo. I set the global store to an unused $PATH bin.

Sorry for the screenshot, StackBlitz sucks on phones and I'm not vertical yet.

I'll publish a stable tag for this purpose with extra heavy petting, I'll write some more tests this week. Use prod to make it fast so pnpm i -g -P g4c@stable and do the same for @antfu/ni

The big thing we're missing, that we can now do is a create/fork/new link.

vslite.com/~/github.com/kat-tax/vslite.git

Should fork out of the box... And run. Let's worry about confused sheriffs and secrets next thing. You could even charge for a reverse proxy and 🤑

If you do that I'll work on making

vslite.com/~/gh/vitejs/vite/tree/main/packages/create-vite/template-vue-ts.

Work like the server side weirdness of vite.new/vue-ts

[mindplay-dk]

It clones to the current directory... so if there's a conflict... like package.json it safely exits.

Ah, I see, well, the headline says "basic pure js git CLI implementation", so I assumed this would be compatible with git.

And isomorphic-git itself comes with an isogit, which isn't compatible with git either.

So, forgive me for asking, but what is the goal with g4c when isogit already exists?

[FossPrime]

And isomorphic-git itself comes with an isogit, which isn't compatible with git either.

Nobody wants to do this isogit clone --url=https://github.com/isomorphic-git/isomorphic-git --depth=1 --singleBranch

goals

1. make common git commands easy to use in javascript shells
2. give configuration options similar to prettier to make it easier to use in projects with package.json files.
3. Minimize cold-boot time and overhead in browser environments
4. Improve DX for javascript developers as much as possible

I will pass through commands I haven't implemented to isogit... not sure how helpful that would be.

[TheUltDev]

@FossPrime I pushed the ability to auto-clone and install via a url:
https://vslite.dev/~/github.com/kat-tax/vslite.git

Auto-running will require a bit more work, reading the package.json and looking for stackblitz config start command or for common npm script names like start then dev, etc... will do that later.

[FossPrime]

Auto-running will require a bit more work, reading the package.json and looking for stackblitz config start command or for common npm script names like start then dev, etc... will do that later.

g4c supports an env var that I use to hide secrets in Replit from .git and prying eyes.
If you set the env var I could add autoRun and autoChdir to it, and you can prepend the env var however you see fit.

chdir is a node name... I don't make the rules: https://nodejs.org/api/process.html#processchdirdirectory

This would also help because URL's will get tricky and resolving the name of the repo or package name is not trivial. I could also add cloneToPkgName so that I mv the cloned dir to the package.json name, rather than the git repo's name. This is a big deal with monorepos the very popular monorepo structure.

I don't know where that exitCode error is coming from.

There is also a huge WebContainer update that came out today. Finally, I know somethings about ESM that may be of help to you... ... ...

https://stackblitz.com/edit/isstackblitz?file=index.js

https://stackblitz.com/edit/fsh?file=README.md

---

I could also add a way to execute arbitrary Js code in a new Function(...) after success. Executing a new node instance takes like 100 milliseconds.

https://stackblitz.com/edit/vitejs-vite-itctul?file=package.json&terminal=m,y,%20,s,e,c,r,e,t,%20,m,e,s,s,a,g,e,dev

I also have to process package.json anyway for commit's author name and author email and other config.

---

Non exhaustive list of URL's I'm trying to support:

Coming soon

- From sub folders
  - https://github.com/vitejs/vite/tree/main/packages/create-vite/template-vanilla-ts
- From the pull request URL's
  - https://github.com/marshallswain/feathers-pinia/pull/132
 - From a folder with a branch that has forward slashes
  - https://github.com/vitejs/vite/tree/docs/philosophy/packages/create-vite/template-vanilla-ts
- From commit hash
  - https://github.com/joshmarinacci/node-pureimage/commit/c9aff9dbb39d06d5dca86c9a375dd86c3cea0315
- From version tags
  - https://github.com/marshallswain/feathers-pinia/releases/tag/v3.0.0
- ✅ Extensionless .git url
  - https://github.com/dominictarr/JSONStream
- ✅ With .git extension
  - git://github.com/feathersjs/feathers.git
- ✅ Cloning from private repos
  - https://vslite.dev/~/gitlab+deploy-token-2130493:[email protected]/vblip/example-private.git
- ❗From glitch password-less, username'd repo url
  - [email protected]/git/friggin-cat-weather.git
- ❓GitLab Wiki URL's
  - https://gitlab.com/painlessMesh/painlessMesh.wiki.git
- Starts with double protocol
  - https://https://gitlab.com/painlessMesh/painlessMesh.wiki.git

Later

- Git branch references, supported by git and yarn
  - ssh://github.com/user/repo.git**#abcd1234**
- SSH to https conversion, if in web container
  - [email protected]:sindresorhus/pkg-conf.git
- With .git extension
  - git://github.com/feathersjs/feathers.git
- From Google Cloud Source with header based cookie authentication
- From Github OAuth CLI authentication

[FossPrime]

I noticed you're using pnpx ... the performance benefits are not existent with single packages like ni and g4c.

More importantly... there is the concern of some tomfoolery with hijacking the source package for a malicious one. npx supports -- for this purpose.

Also, cloning in place seems more appropriate... as this is what you get on most all other sandboxes... rather than being down two floors of $HOME. I'll think of a way to specify clone in place. Perhaps setting the clone dir to ../. So git clone repo_url '../'

[TheUltDev]

@FossPrime so I switched to pnpm dlx because as of the release today WebContainers has native npx support. So when this happened, it now prompts you to install needed packages. You have to enter y to install dependencies. This fucks with the git repo auto-install from url. PNPM doesn't have this prompt, so I used it instead. If you can find a way to work around that, feel free to change it to NPX.

[FossPrime]

@FossPrime so I switched to pnpm dlx because as of the release today WebContainers has native npx support. So when this happened, it now prompts you to install needed packages. You have to enter y to install dependencies. This fucks with the git repo auto-install from url. PNPM doesn't have this prompt, so I used it instead. If you can find a way to work around that, feel free to change it to NPX.

The modem way is npx -y. If it's running in a non interactive environment it will not prompt. Finally if you must support old versions of non too, you can do npm_config_yes=true npx

I recommend npx -y as WebContainers were the last place that didn't support it before yesterday.
npm/cli#1935 (comment)

In my Replit environments I always use npm_config_yes=true and its never a problem.

You can also pipe output somewhere, including dev null, and it will stop beimg interactive.

[TheUltDev]

@FossPrime npx -y does not work. I tried it to no avail. Will attempt it again later though.

[FossPrime]

@FossPrime npx -y does not work. I tried it to no avail. Will attempt it again later though.

As of yesterday it does. npm's argument parsing might be the culprit
for weirdness -- solves that. Check your cache? It should have NPM v9

[TheUltDev]

@FossPrime you're right, it works now. Just pushed an update.

Also updated to clone to the root, not a subfolder.

[TheUltDev]

Closing this out since we have really good support now.

Anything else can be done in another task.

@FossPrime thanks for driving this.

[mindplay-dk]

@TheUltDev having npm support is really cool, but my request was for git support 😊

I wanted this mainly so I could install from git: sources, but wouldn't it be cool if you could just open a tab an git clone like usual?

I guess that would require a proxy server and stuff though, which I guess makes it more server dependent. Man, I wish there were some way to make this entirely client-side - it seems to silly, having to set up a proxy server and waste all this bandwidth, just so the browser can be allowed to make requests to servers. I mean, I understand why, but still, seems silly for something like this.

Would be nice if you could just prompt the user for permission to make requests to other domains.

Could be done with a chrome extension, I suppose. But not with any browser feature I can think of?

Or I guess with FileSystem API you could clone/commit to the user's local filesystem, but then that's sort of a different feature, isn't it.

And either way, neither g4c or isogit are git compatible, so maybe this feature is too difficult to add. 🤔

[FossPrime]

I wanted this mainly so I could install from git: sources, but wouldn't it be cool if you could just open a tab an git clone like usual?

A chrome extension like this might be nice https://github.com/ptoner/isomorphic-git-cors-extension

You can run git clone like normal... what call do you need that you can't make right now?

[mindplay-dk]

You can run git clone like normal... what call do you need that you can't make right now?

git clone {url} didn't work for me? it requires git clone --url={url} it seems?

As said, to be able to npm install with git: dependencies, CLI syntax would need to be compatible, which appears to be a non-goal for both projects.

Either way, this is all moot unless this is actually a feature the author wants in this tool. I just thought it would be cool. 🙂

[TheUltDev]

@mindplay-dk git clone <url> is working for me. Could you provide details of the error you are seeing?

[mindplay-dk]

@TheUltDev sorry, I was confused - I didn't think it was actually in the deployed vslite.dev yet, you're right, it does work! 😊

I noticed one little bug though - after cloning, only .git and not the actual files are visible in the tree:

[mindplay-dk]

(for that matter, you could probably hide .git from the tree - VS code doesn't do that, which is kind of annoying.)

[TheUltDev]

@mindplay-dk awesome, glad to hear it works for you. Concerning the file tree, it's being refactored in #6. It's basically there as a proof of concept right now.