Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deployment progress events from the backend #209

Merged
merged 11 commits into from
Sep 15, 2023
Merged

Deployment progress events from the backend #209

merged 11 commits into from
Sep 15, 2023

Conversation

mmarchetti
Copy link
Contributor

Resolves #202

Intent

  • Sends progress events during R and Python package restores
  • Cleans up duplicate attributes in event messages

Type of Change

  • Bug Fix
  • New Feature
  • Breaking Change

Automated Tests

Directions for Reviewers

Deploy a Python app that requires a new environment to be built, and one that doesn't. Verify logs/event stream contains events corresponding to the package downloads/installs in the Connect app log. Note that Python packages don't always log download or install messages, depending on whether the package was in the download cache, or is present in the base virtualenv with the same or different version.

@mmarchetti mmarchetti marked this pull request as ready for review September 12, 2023 16:22
Copy link
Collaborator

@tdstein tdstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functionally LGTM. Just nit questions about Go conventions.

internal/logging/logger.go Outdated Show resolved Hide resolved
internal/logging/logger.go Outdated Show resolved Hide resolved
internal/logging/logger.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@kgartland-rstudio kgartland-rstudio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified! Now seeing information about Python environments on Connect:

Fresh environment deployment:

SSE will send: {"Time":"2023-09-15T15:01:46.98406-04:00","Type":"publish/restorePythonEnv/log","Data":{"Level":"INFO","Message":"2023/09/15 19:01:46.620758558 Creating environment: g62yybu96Nm-42QMs6oLZw","source":"serverLog"}}
time=2023-09-15T15:01:51.729-04:00 level=INFO msg="2023/09/15 19:01:50.992817249 Collecting pip" source=serverLog event_op=publish/restorePythonEnv
SSE will send: {"Time":"2023-09-15T15:01:51.729006-04:00","Type":"publish/restorePythonEnv/log","Data":{"Level":"INFO","Message":"2023/09/15 19:01:50.992817249 Collecting pip","source":"serverLog"}}
time=2023-09-15T15:01:51.729-04:00 level=INFO msg="2023/09/15 19:01:51.119678062 Using cached https://files.pythonhosted.org/packages/50/c2/e06851e8cc28dcad7c155f4753da8833ac06a5c704c109313b8d5a62968a/pip-23.2.1-py3-none-any.whl" source=serverLog event_op=publish/restorePythonEnv
SSE will send: {"Time":"2023-09-15T15:01:51.729156-04:00","Type":"publish/restorePythonEnv/log","Data":{"Level":"INFO","Message":"2023/09/15 19:01:51.119678062 Using cached https://files.pythonhosted.org/packages/50/c2/e06851e8cc28dcad7c155f4753da8833ac06a5c704c109313b8d5a62968a/pip-23.2.1-py3-none-any.whl","source":"serverLog"}}
time=2023-09-15T15:01:52.350-04:00 level=INFO msg="2023/09/15 19:01:51.756526808 Collecting setuptools" source=serverLog event_op=publish/restorePythonEnv

Cached Environment Deployment:

SSE will send: {"Time":"2023-09-15T14:45:10.25166-04:00","Type":"publish/restorePythonEnv/log","Data":{"Level":"INFO","Message":"2023/09/15 18:45:10.089103994 Using cached environment: Pa5tKYxKMSNr6xGrmbr8Nw","source":"serverLog"}}
time=2023-09-15T14:45:11.492-04:00 level=INFO msg="2023/09/15 18:45:11.455331723 Packages in the environment: aiofiles==23.2.1, anyio==3.6.2, appdirs==1.4.4, asgiref==3.6.0, certi
requests
fi==2020.12.5, click==8.1.3, distlib, fastapi==0.95.2, filelock, h11==0.14.0, idna==3.4, pydantic==1.10.7, PyJWT==2.7.0, semver==2.13.0, six==1.16.0, sniffio==1.3.0, starlette==0.27.0, tornado, typing_extensions==4.5.0, uvicorn==0.22.0, virtualenv, websockets==11.0.3, " source=serverLog event_op=publish/restorePythonEnv
SSE will send: {"Time":"2023-09-15T14:45:11.492115-04:00","Type":"publish/restorePythonEnv/log","Data":{"Level":"INFO","Message":"2023/09/15 18:45:11.455331723 Packages in the environment: aiofiles==23.2.1, anyio==3.6.2, appdirs==1.4.4, asgiref==3.6.0, certifi==2020.12.5, click==8.1.3, distlib, fastapi==0.95.2, filelock, h11==0.14.0, idna==3.4, pydantic==1.10.7, PyJWT==2.7.0, semver==2.13.0, six==1.16.0, sniffio==1.3.0, starlette==0.27.0, tornado, typing_extensions==4.5.0, uvicorn==0.22.0, virtualenv, websockets==11.0.3, ","source":"serverLog"}}
time=2023-09-15T14:45:11.492-04:00 level=INFO msg="2023/09/15 18:45:11.456554876 Creating lockfile: python/requirements.txt.lock" source=serverLog event_op=publish/restorePythonEnv

@mmarchetti mmarchetti merged commit be18719 into main Sep 15, 2023
13 checks passed
@mmarchetti mmarchetti deleted the mm-more-events branch September 15, 2023 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Send progress updates to the event stream to describe intermediate progress
3 participants