Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new Jupyter kernel supervisor #4910

Merged
merged 93 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from 90 commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
ccf788f
stubs for kallichore adapter extension
jmcphers Sep 18, 2024
1ab999c
port install-kernel => install-kallichore
jmcphers Sep 18, 2024
1c9ad13
tweak kallichore install script; ignore binaries
jmcphers Sep 18, 2024
4a71bd8
bumpt to kallichore 0.1.2
jmcphers Sep 18, 2024
b8edc3a
Merge remote-tracking branch 'origin/main' into feature/kallichore-ad…
jmcphers Sep 18, 2024
f8b63be
start kcserver at startup
jmcphers Sep 19, 2024
55cb7ab
initial codegen'ed api client
jmcphers Sep 19, 2024
3dcbefe
pick an unused port to run kallichore
jmcphers Sep 19, 2024
eea2b2c
factor API wrapper into its own class; expose on extension
jmcphers Sep 19, 2024
6075897
toggle for using kallichore to start R
jmcphers Sep 19, 2024
3034530
add session adapter
jmcphers Sep 19, 2024
a515149
add websocket; connect after started
jmcphers Sep 20, 2024
b253eb9
allow for sideloading kallichore from adjacent repository
jmcphers Sep 20, 2024
ee03621
logic for ascertaining working directory
jmcphers Sep 20, 2024
94794e8
start stubbing out protocol message types
jmcphers Sep 20, 2024
6a45d79
forward kernel status messages
jmcphers Sep 20, 2024
8f85dc8
plumbing basic kernel info request
jmcphers Sep 20, 2024
5beca25
more types & RPC flow sketch
jmcphers Sep 23, 2024
b8f6d7b
execute request types; complete plumbing for rpc
jmcphers Sep 23, 2024
81568f3
basic iscomplete/execute flow
jmcphers Sep 23, 2024
f92a2f5
Merge remote-tracking branch 'origin/main' into feature/kallichore-ad…
jmcphers Sep 23, 2024
72d4fd0
start handling output types
jmcphers Sep 23, 2024
4a78919
execute input / kernel status messages
jmcphers Sep 23, 2024
78fd86f
impl client list RPC
jmcphers Sep 23, 2024
d68600d
reconnect to existing kallichore server in workspace
jmcphers Sep 23, 2024
cd4987b
add a script for regenerating the api sources
jmcphers Sep 24, 2024
41262f7
regenerate API with active session support
jmcphers Sep 24, 2024
a899cf9
impl reconnect
jmcphers Sep 24, 2024
f3b42ad
impl comm open command
jmcphers Sep 24, 2024
835ad58
impl comm close
jmcphers Sep 24, 2024
e20287d
impl start dap/lsp (WIP)
jmcphers Sep 24, 2024
ce145ad
move laborious message translation to its own class
jmcphers Sep 24, 2024
95fed67
impl handlers for some less common message types
jmcphers Sep 24, 2024
ff8aa2c
impl send client message
jmcphers Sep 24, 2024
a65e82e
impl shutdown, interrupt
jmcphers Sep 24, 2024
02773bc
wait for ready state before getting kernel info
jmcphers Sep 24, 2024
5aef2fb
create log per kernel
jmcphers Sep 25, 2024
73d9a91
stream log file to output channel on startup
jmcphers Sep 25, 2024
db34549
fetch session from Kallichore before restore; re-establish logs
jmcphers Sep 25, 2024
55368b1
some initial work on restart
jmcphers Sep 25, 2024
6fd0c24
import restart method
jmcphers Sep 27, 2024
c491061
use api to restart session
jmcphers Sep 27, 2024
fab8fef
move willstart event to be extension-driven
jmcphers Sep 27, 2024
8cd59eb
use correct values for interrupt mode & username
jmcphers Sep 27, 2024
9370482
plumbing for input and rpc replies
jmcphers Sep 27, 2024
6a29a76
let python sessions opt in to kallichore
jmcphers Sep 27, 2024
f61a740
codegen for shutdown server method
jmcphers Sep 30, 2024
fb6334a
impl ui comm call_method for R
jmcphers Sep 30, 2024
08b7667
impl Debug Protocol Adapter
jmcphers Oct 1, 2024
60021da
Merge remote-tracking branch 'origin/main' into feature/kallichore-ad…
jmcphers Oct 1, 2024
ce3bdc2
bring in new jupyter adapter types; fix comment
jmcphers Oct 1, 2024
ea1c971
port new logging behavior from jupyter adapter
jmcphers Oct 1, 2024
3e60c38
make create/restore APIs async
jmcphers Oct 1, 2024
abd72e5
defend against hangs with timeouts; add comments
jmcphers Oct 1, 2024
4b724f5
impl show log channels, factor out msg id gen
jmcphers Oct 1, 2024
6738026
wait for idle before reconnecting if kernel is busy
jmcphers Oct 1, 2024
6528e75
add support for passing metadata when opening a comm
jmcphers Oct 1, 2024
98d500f
regen api, create bearer token
jmcphers Oct 2, 2024
3716f3d
bump kallichore version
jmcphers Oct 2, 2024
d868a42
re-establish bearer token on reconnect
jmcphers Oct 3, 2024
5b53ee0
cleanup all websocket connections on exit
jmcphers Oct 3, 2024
6a1b037
externalize native dependencies
jmcphers Oct 3, 2024
8ab51a7
tighten up startup sequence; more docs
jmcphers Oct 3, 2024
2156b74
faster and more resilient startup; doc folders
jmcphers Oct 3, 2024
4645529
call kallichore server install script during extension build
jmcphers Oct 3, 2024
5003f01
pull dynamic state from server instead of guessing
jmcphers Oct 3, 2024
e27c316
more docs; check --logfile for python
jmcphers Oct 3, 2024
e6c0cd9
make log level configurable
jmcphers Oct 3, 2024
2985f34
Merge remote-tracking branch 'origin/main' into feature/kallichore-ad…
jmcphers Oct 3, 2024
3a02c71
handle websocket close events
jmcphers Oct 3, 2024
73b7f76
wrap socket in a class so we can attach user/session id
jmcphers Oct 4, 2024
953997c
ensure all send errors are handled
jmcphers Oct 4, 2024
64ff6f4
use signal interrupts on Windows
jmcphers Oct 4, 2024
2aa760b
pass additional environment variables
jmcphers Oct 4, 2024
95ee5ff
place output from kernel in the Kernel output channel
jmcphers Oct 4, 2024
062a984
infrastructure for tests
jmcphers Oct 4, 2024
9cda660
add dedicated test command
jmcphers Oct 4, 2024
44eca2f
regen API with server status
jmcphers Oct 4, 2024
6d0c223
basic start server / get version test
jmcphers Oct 4, 2024
f80ab96
don't eagerly start the server unless configured to do so
jmcphers Oct 4, 2024
7645930
wait for LSP to start before resolving promise
jmcphers Oct 4, 2024
dc9bd27
remove embedded kallichore adapter typedefs
jmcphers Oct 4, 2024
bf181bd
use python extension's TS formatting rules
jmcphers Oct 4, 2024
c439a89
skip kallichore tests on pull requests due to public repo
jmcphers Oct 4, 2024
1075b0f
refuse to connect to a session that has exited
jmcphers Oct 5, 2024
b586a29
bump verison of kallichore server
jmcphers Oct 7, 2024
55a7873
code review feedback
jmcphers Oct 7, 2024
b9a666a
bump kallichore version
jmcphers Oct 7, 2024
918e1d0
add support for base64 encoded binary buffers
jmcphers Oct 7, 2024
4aab448
impl attachonstartup/sleeponstartup (for ark)
jmcphers Oct 8, 2024
74b65c2
skip download on all public repository CI jobs
jmcphers Oct 8, 2024
fb7f93d
add universal app exclusions for native modules
jmcphers Oct 8, 2024
9f27090
restore kerberos module
jmcphers Oct 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@
**/extensions/html-language-features/server/lib/jquery.d.ts
**/extensions/html-language-features/server/src/test/pathCompletionFixtures/**
**/extensions/ipynb/notebook-out/**
# --- Start Positron ---
# Ignore OpenAPI generated files
**/extensions/kallichore-adapter/src/kcclient/**
# --- End Positron ---
**/extensions/markdown-language-features/media/**
**/extensions/markdown-language-features/notebook-out/**
**/extensions/markdown-math/notebook-out/**
Expand Down
5 changes: 5 additions & 0 deletions .vscode-test.js
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,11 @@ const extensions = [
workspaceFolder: 'extensions/positron-run-app/test-workspace',
mocha: { timeout: 60_000 }
},
{
label: 'kallichore-adapter',
workspaceFolder: path.join(os.tmpdir(), `kallichore-adapter-${Math.floor(Math.random() * 100000)}`),
mocha: { timeout: 60_000 }
},
// --- End Positron ---
{
label: 'microsoft-authentication',
Expand Down
12 changes: 11 additions & 1 deletion build/filters.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,19 @@ module.exports.all = [
'!**/node_modules/**',

// --- Start Positron ---
// Excluded since it's generated code (an OpenAPI client)
'!extensions/kallichore-adapter/src/kcclient/**/*',

// Excluded since it comes from an external source with its own hygiene
// rules
'!extensions/positron-python/**/*',

// Excluded since it comes from an external source with its own hygiene
// rules
'!extensions/open-remote-ssh/**/*',
'!test/smoke/test-repo/**/*'

// Excluded since it isn't shipping code
'!test/smoke/test-repo/**/*',
// --- End Positron ---
];

Expand Down
1 change: 1 addition & 0 deletions build/gulpfile.extensions.js
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ const ext = require('./lib/extensions');
// });
const compilations = [
// --- Start Positron ---
'extensions/kallichore-adapter/tsconfig.json',
'extensions/open-remote-ssh/tsconfig.json',
'extensions/positron-code-cells/tsconfig.json',
'extensions/positron-connections/tsconfig.json',
Expand Down
1 change: 1 addition & 0 deletions build/npm/dirs.js
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ const dirs = [
'extensions/json-language-features/server',
// --- Start Positron ---
'extensions/jupyter-adapter',
'extensions/kallichore-adapter',
// --- End Positron ---
'extensions/markdown-language-features',
'extensions/markdown-math',
Expand Down
10 changes: 5 additions & 5 deletions extensions/jupyter-adapter/src/Api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,16 @@ export class JupyterAdapterApiImpl implements JupyterAdapterApi {
kernel: JupyterKernelSpec,
dynState: positron.LanguageRuntimeDynState,
extra: JupyterKernelExtra,
): JupyterLanguageRuntimeSession {
return new LanguageRuntimeSessionAdapter(
): Promise<JupyterLanguageRuntimeSession> {
return Promise.resolve(new LanguageRuntimeSessionAdapter(
runtimeMetadata,
sessionMetadata,
this._context,
this._channel,
kernel,
dynState,
extra
);
));
}

/**
Expand All @@ -59,7 +59,7 @@ export class JupyterAdapterApiImpl implements JupyterAdapterApi {
restoreSession(
runtimeMetadata: positron.LanguageRuntimeMetadata,
sessionMetadata: positron.RuntimeSessionMetadata
): JupyterLanguageRuntimeSession {
): Promise<JupyterLanguageRuntimeSession> {

// Get the serialized session from the workspace state. This state
// contains the information we need to reconnect to the session, such as
Expand Down Expand Up @@ -90,7 +90,7 @@ export class JupyterAdapterApiImpl implements JupyterAdapterApi {
// happens later when the session is started.
adapter.restoreSession(serialized.sessionState);

return adapter;
return Promise.resolve(adapter);
}

/**
Expand Down
4 changes: 2 additions & 2 deletions extensions/jupyter-adapter/src/jupyter-adapter.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ export interface JupyterAdapterApi extends vscode.Disposable {
kernel: JupyterKernelSpec,
dynState: positron.LanguageRuntimeDynState,
extra?: JupyterKernelExtra | undefined,
): JupyterLanguageRuntimeSession;
): Promise<JupyterLanguageRuntimeSession>;

/**
* Restore a session for a Jupyter-compatible kernel.
Expand All @@ -161,7 +161,7 @@ export interface JupyterAdapterApi extends vscode.Disposable {
restoreSession(
runtimeMetadata: positron.LanguageRuntimeMetadata,
sessionMetadata: positron.RuntimeSessionMetadata
): JupyterLanguageRuntimeSession;
): Promise<JupyterLanguageRuntimeSession>;

/**
* Finds an available TCP port for a server
Expand Down
25 changes: 25 additions & 0 deletions extensions/kallichore-adapter/extension.webpack.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/*---------------------------------------------------------------------------------------------
* Copyright (C) 2024 Posit Software, PBC. All rights reserved.
* Licensed under the Elastic License 2.0. See LICENSE.txt for license information.
*--------------------------------------------------------------------------------------------*/

//@ts-check

'use strict';

const path = require('path');
const withDefaults = require('../shared.webpack.config');

module.exports = withDefaults({
context: __dirname,
entry: {
extension: './src/extension.ts',
},
node: {
__dirname: false
},
externals: {
'bufferutil': 'commonjs bufferutil',
'utf-8-validate': 'commonjs utf-8-validate'
}
});
Loading
Loading