fix: duplicate read proto #1737

Shylock-Hg · 2024-04-16T06:07:35Z

Summary:
This PR fixes related issue by avoiding double read file, the second read proto file will use the content of the first directly.

Current output:

shylock@localhost:~/Workspace/tailcall> ./target/debug/tailcall start examples/grpc.graphql 
 INFO File read: examples/grpc.graphql ... ok
 INFO File read: examples/../src/grpc/tests/proto/news.proto ... ok
 INFO N + 1 detected: 0
 INFO 🚀 Tailcall launched at [127.0.0.1:8000] over HTTP/1.1
 INFO 🌍 Playground: http://127.0.0.1:8000

Issue Reference(s):
Fixes #... (Replace "..." with the issue number)
Close #1734
/claim #1734

Build & Testing:

I ran cargo test successfully.
I have run ./lint.sh --mode=fix to fix all linting issues raised by ./lint.sh --mode=check.

Checklist:

I have added relevant unit & integration tests.
I have updated the documentation accordingly.
I have performed a self-review of my code.
PR follows the naming convention of <type>(<optional scope>): <title>

Summary by CodeRabbit

Refactor
- Optimized memory usage in configuration reading by using references instead of owned values.
- Improved content reading and file descriptor processing in ProtoReader for enhanced performance and consistency.

coderabbitai · 2024-04-16T06:07:43Z

Walkthrough

The recent updates in the codebase include optimizations in memory management by using references in the ConfigReader and structural improvements in ProtoReader by adding a new method for direct content reading and refactoring the file descriptor processing logic. These changes enhance efficiency and maintainability.

Changes

Files	Changes Summary
`src/config/reader.rs`, `src/proto_reader.rs`	Optimized memory usage by using references, added `read_content` method, and refactored file descriptor processing into `process_file_dp`.

Assessment against linked issues

Objective	Addressed	Explanation
Avoid duplicated CLI output for read files (#1734)	❌	The changes do not address the issue of duplicated CLI outputs directly.

Possibly related issues

Issue refactor: create proto_reader #1619: This issue discusses refactoring to create a separate proto_reader. The changes in this PR, particularly the addition of read_content and refactoring in proto_reader, align with the objectives of creating a more focused and efficient proto reading logic. This suggests that the current PR could be contributing towards the goals of issue refactor: create proto_reader #1619.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

codecov · 2024-04-16T06:13:12Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.14%. Comparing base (4553eec) to head (090b43a).

❗ Current head 090b43a differs from pull request most recent head 65614e2. Consider uploading reports for the commit 65614e2 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1737      +/-   ##
==========================================
+ Coverage   86.93%   87.14%   +0.21%     
==========================================
  Files         155      153       -2     
  Lines       15540    15610      +70     
==========================================
+ Hits        13509    13604      +95     
+ Misses       2031     2006      -25

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

tusharmath

Can u update the description of the PR with the changes that fixed the issue, thanks!

Shylock-Hg · 2024-04-16T06:22:06Z

Can u update the description of the PR with the changes that fixed the issue, thanks!

Ok, it's done.

Shylock-Hg · 2024-04-16T08:22:45Z

Walkthrough

The recent updates in the codebase include optimizations in memory management by using references in the ConfigReader and structural improvements in ProtoReader by adding a new method for direct content reading and refactoring the file descriptor processing logic. These changes enhance efficiency and maintainability.

Changes

Files Changes Summary
src/config/reader.rs, src/proto_reader.rs Optimized memory usage by using references, added read_content method, and refactored file descriptor processing into process_file_dp.

Assessment against linked issues

Objective Addressed Explanation
Avoid duplicated CLI output for read files (#1734) ❌ The changes do not address the issue of duplicated CLI outputs directly.

Possibly related issues

Issue refactor: create proto_reader #1619: This issue discusses refactoring to create a separate proto_reader. The changes in this PR, particularly the addition of read_content and refactoring in proto_reader, align with the objectives of creating a more focused and efficient proto reading logic. This suggests that the current PR could be contributing towards the goals of issue refactor: create proto_reader #1619.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

The issue is addressed:

shylock@localhost:~/Workspace/tailcall> ./target/debug/tailcall start examples/grpc.graphql 
 INFO File read: examples/grpc.graphql ... ok
 INFO File read: examples/../src/grpc/tests/proto/news.proto ... ok
 INFO N + 1 detected: 0
 INFO 🚀 Tailcall launched at [127.0.0.1:8000] over HTTP/1.1
 INFO 🌍 Playground: http://127.0.0.1:8000

tusharmath · 2024-04-16T08:30:17Z

src/proto_reader.rs

+        path: &str,
+        file_dp: FileDescriptorProto,
+    ) -> anyhow::Result<ProtoMetadata> {
+        if file_dp.package.is_none() {


I think it would be better to build caching inside of ResourceReader, something like:

ResourceReader::init(runtime, true)

ResourceReader, could maintain a HashMap of file path and content and cache it locally when enabled.

If follow this, we need change &self of ResouceReader::read_file to &mut self, and all other structures contain it. To fix a small issue, it modifies too much.

And HashMap is not Send, to do it, maybe we need lock it, and it will become bottleneck.

We can wrap it in a Arc of Mutex. This isn't hot code so performance shouldn't be a problem.

And there is another issue, the first reading is called by ConfigReader::resource_reader, the second is called by ConfigReader::proto_reader, they are different object. Maybe we need to put HashMap in ConfigReader`.

Shylock-Hg · 2024-04-16T12:18:00Z

Reopen in #1741

Shylock-Hg added 2 commits April 16, 2024 14:03

Avoid duplicate read proto file.

aabccc1

Lint fix.

afa9662

algora-pbc bot mentioned this pull request Apr 16, 2024

Duplicated CLI output related to read files #1734

Closed

2 tasks

algora-pbc bot added the 🙋 Bounty claim label Apr 16, 2024

github-actions bot added the type: fix Iterations on existing features or infrastructure. label Apr 16, 2024

Shylock-Hg changed the title ~~Fix/duplicate read proto~~ fix: duplicate read proto Apr 16, 2024

Merge branch 'main' into fix/duplicate-read-proto

090b43a

tusharmath reviewed Apr 16, 2024

View reviewed changes

Shylock-Hg requested a review from tusharmath April 16, 2024 06:22

Merge branch 'main' into fix/duplicate-read-proto

65614e2

tusharmath reviewed Apr 16, 2024

View reviewed changes

Shylock-Hg closed this Apr 16, 2024

Shylock-Hg deleted the fix/duplicate-read-proto branch May 7, 2024 10:51

Shylock-Hg restored the fix/duplicate-read-proto branch May 7, 2024 10:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: duplicate read proto #1737

fix: duplicate read proto #1737

Shylock-Hg commented Apr 16, 2024 •

edited

Loading

coderabbitai bot commented Apr 16, 2024 •

edited

Loading

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (`.coderabbit.yaml`)

Documentation and Community

codecov bot commented Apr 16, 2024 •

edited

Loading

tusharmath left a comment

Shylock-Hg commented Apr 16, 2024

Shylock-Hg commented Apr 16, 2024

Walkthrough

Changes

Assessment against linked issues

Possibly related issues

tusharmath Apr 16, 2024

Shylock-Hg Apr 16, 2024

Shylock-Hg Apr 16, 2024

tusharmath Apr 16, 2024

Shylock-Hg Apr 16, 2024 •

edited

Loading

Shylock-Hg commented Apr 16, 2024

fix: duplicate read proto #1737

fix: duplicate read proto #1737

Conversation

Shylock-Hg commented Apr 16, 2024 • edited Loading

Summary by CodeRabbit

coderabbitai bot commented Apr 16, 2024 • edited Loading

Walkthrough

Changes

Assessment against linked issues

Possibly related issues

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (.coderabbit.yaml)

Documentation and Community

codecov bot commented Apr 16, 2024 • edited Loading

Codecov Report

tusharmath left a comment

Choose a reason for hiding this comment

Shylock-Hg commented Apr 16, 2024

Shylock-Hg commented Apr 16, 2024

Walkthrough

Changes

Assessment against linked issues

Possibly related issues

tusharmath Apr 16, 2024

Choose a reason for hiding this comment

Shylock-Hg Apr 16, 2024

Choose a reason for hiding this comment

Shylock-Hg Apr 16, 2024

Choose a reason for hiding this comment

tusharmath Apr 16, 2024

Choose a reason for hiding this comment

Shylock-Hg Apr 16, 2024 • edited Loading

Choose a reason for hiding this comment

Shylock-Hg commented Apr 16, 2024

Shylock-Hg commented Apr 16, 2024 •

edited

Loading

coderabbitai bot commented Apr 16, 2024 •

edited

Loading

CodeRabbit Configration File (`.coderabbit.yaml`)

codecov bot commented Apr 16, 2024 •

edited

Loading

Shylock-Hg Apr 16, 2024 •

edited

Loading