Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make a proper Log class and methods #948

Open
wants to merge 36 commits into
base: v5
Choose a base branch
from
Open

Make a proper Log class and methods #948

wants to merge 36 commits into from

Conversation

plocket
Copy link
Collaborator

@plocket plocket commented Oct 2, 2024

Apparently there was still a while to go. I still have decision and design/architecture docs to write, but this is ready for review. What I most want to discuss is how to handle bugs internal to .debug().

I wish this integrated the report more too, but this is already big enough.

In this PR, I have:

  • Added tests for any new features or bug fixes (if relevant)
  • Added my changes to the CHANGELOG.md at the top, under the "Unreleased" section
  • Ensured issues that this PR closes will be automatically closed

Reason for this PR

The logs have been a bit of a mess for a while. The verbose logs could only be logged live, not saved. Saving to the debug log was a bit add-hoc. There wasn't a good differentiation between console logs and debug logs. This addresses a few of those problems.

Links to any solved or related issues

Close #805, create verbose-type log
Close #924, remove extra url print
Close #659, abstract adding to debug_log
Close #925, allow a Log to throw an error
Address #461, setup and takedown reports

Any manual testing I have done to ensure my PR is working

Visually checked various log files.

plocket and others added 8 commits August 13, 2024 12:25
- Put a wait in time.js
- Modified review doc based on more findings
- Ask questions
- Experiment with missing logs on error. See https://replit.com/@plocket/Avoid-leaving-a-consoleConsole-file-blank-on-error#index.js
- Describe new log-calling signature
- Re-throw errors
- Preserve log levels in debug console
- Use `recursive` when creating artifacts folder

Co-authored-by: will-morningstar <[email protected]>
Co-authored-by: cthulahoops <[email protected]>
Copy link
Collaborator

@BryceStevenWilley BryceStevenWilley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial thoughts. Didn't get a chance to look through everything, but focused on the design docs and how the new log API is used in Kiln itself.

`error`:
- Avoid using `error` explicitly. Only the debug logs should use the `error` level, and only to record critical errors that come in with `throw`.
`info`:
- Log expected errors as `info`. For example, the aborted promises when awaiting navigation.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really interesting rule for what to use when. Trying to sit with my biases of what INFO is usually used for, and determine if it's worth renaming INFO to something else, like "suspicious" (SUS). Something that, by itself, can be ignored (and doesn't need to be investigated), but with a failure, might be an issue. Usually I'd use warn for that, but I do like warn being only for user focused logs

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this line of thought. No need to reply if you don't have bandwidth, I'll just brainstorm before.

Non-alarming words that can have an internal meaning:

  • note
  • detail
  • cue
  • queue
  • data
  • trace
  • dig
  • finding
  • memo
  • review
  • intel
  • mark
  • comms
  • signal
  • task
  • todo

Somewhat alarming words that can have an internal meaning:

  • suspicious
  • sus
  • alert
  • notice
  • clue
  • mark
  • symptom
  • advise
  • caution

We could also require using "info" for things that don't need attention and leave "debug" as our internal signal to examine further. Or maybe, going in a different direction - just "alkiln"/"alk".

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Misread this note a bit. I think an alternative word (like "SUS") would be useful for internal warnings, but just want to be clear that an aborted promise is expected and is really just information - when one promise wins a race, the others will, of course, be (correctly) aborted. I added a design doc about the various meanings of "error" that might help clarify this a bit. I sort of want to combine that doc with this one, but I haven't seen how to make it clear.

lib/docassemble/server_install.js Outdated Show resolved Hide resolved
docs/decisions/reports_and_logs_2024_08_11.md Outdated Show resolved Hide resolved
lib/docassemble/docassemble_api_interface.js Show resolved Hide resolved
lib/docassemble/docassemble_api_interface.js Outdated Show resolved Hide resolved
## Non/anti-goals

- Spend a month on this
- Create a complex and heavy system to try to abstract absolutely everything
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An additional non-goal that I think you might have; logs that are only understandable if you read the surrounding code. A lot of things can be simplified if you assume that, but it def raises the effort level needed by users quite a bit

@plocket
Copy link
Collaborator Author

plocket commented Oct 16, 2024

Didn't get a chance to look through everything, but focused on the design docs and how the new log API is used in Kiln itself.

Don't worry about looking at the code implementation for now. I'm doing a fair amount of refactoring. Thanks, @BryceStevenWilley!

@plocket
Copy link
Collaborator Author

plocket commented Oct 25, 2024

Ready, barring 1 main discussion point (and the docs). That one item - how to handle bugs internal to .debug(). They bring up the potential for infinite loops under very specific circumstances. The max_loops variable is a safety valve right now, but I hate it.

Thoughts:

  1. I really don't like using max_loops or some other kind of cut-off. If we use it, we need to make it high enough that useful information doesn't get missed. That said, it just smells bad. Calling .debug() internally is just asking for trouble, even if we're calling it with what we believe to be safe arguments.
  2. Option - collect the unexpected behavior messages as a list and, at the end, decide whether to save them to the debug log or to console log them. I'm not sure how to pass that data around, though.
  3. Option - console log all unexpected behavior. I'm really reluctant to create that much noise for the authors when it's not-critical functionality.

While we mull over that, I'm going to keep plugging away at the design/decision docs.

@plocket
Copy link
Collaborator Author

plocket commented Oct 28, 2024

My current experiments to eliminate inner calls to .debug() is to replace that call to use a function that abstracts saving straight to the debug file.

plocket and others added 12 commits November 21, 2024 10:36
Also:
* Improve some logs
* Implement logs in unit tests
- Add report_log.txt
- Implement action output (untested)
- Implement logs in unit tests

Co-authored-by: will-morningstar <[email protected]>
- .console and .debug signatures diverge. Reconsider.
- Always throw when calling `.throw`

TODO: When we throw `null` we seem to not get a call stack. We
gotta do something about that. Maybe always make an Error if
`do_throw` is true

Co-authored-by: will-morningstar <[email protected]>
tests, commit before error refactor

Co-authored-by: will-morningstar <[email protected]>
reports, warn in debug on error, catch more.

Note: writing to debug will write to unexpected
writing to unexpected file private, refactor to remove extra try/catches, match up log codes

Tests: Try to neutralize console. Unable. Add new tests, add check string exclusion.

Add notes
plocket and others added 5 commits November 21, 2024 10:50
- Ensure objects don't always log with a leading new line
- Unify language around caught behavior/misbehavior to "Skipped"
- Protect stdout
- Tweak file descriptions (thanks Ethan!)

Co-authored-by: ethanstrominger <[email protected]>
plocket and others added 9 commits November 21, 2024 10:56
* Add decision doc for getting and saving logs

Thanks, @jder, for some extra experimentation with
`console.Console` and write streams.

* Describe the 4 environments in which ALKiln runs, tweak function
descriptions in related files.
* Update Log notes and descriptions
* Tweak and clean up Log descriptions/comments, add TODOs
* Focus log level choice doc on log levels instead of Log
* Clarify types of errors and what to do with them. Dedup some info
* Add more notes and content about logs

Co-authored-by: jder <[email protected]>
* Continue filling documentation
* Describe flow for artifact folder names in 4 environments
* Clarify various docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants