Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to manually enable indexer #290

Merged
merged 1 commit into from
Nov 26, 2024

Conversation

neekolas
Copy link
Contributor

@neekolas neekolas commented Nov 25, 2024

tl;dr

  • Makes the blockchain indexer optional, so that we can only run it on the same instance that runs the sync worker
  • This will require an infra change to set the config

Summary by CodeRabbit

  • New Features
    • Introduced a new configuration option to enable or disable the indexer, enhancing user control over system resources.
  • Bug Fixes
    • Implemented conditional logic to prevent unnecessary indexer initialization, improving resource management and reducing potential errors.

Copy link

coderabbitai bot commented Nov 25, 2024

Walkthrough

The changes introduce a new struct called IndexerOptions in the pkg/config/options.go file, which allows for enabling or disabling an indexer through a boolean field. This struct is integrated into the existing ServerOptions struct. Additionally, the NewReplicationServer function in pkg/server/server.go is modified to conditionally initialize and start the indexer based on the value of options.Indexer.Enable, preventing unnecessary resource usage if the indexer is disabled.

Changes

File Change Summary
pkg/config/options.go Added IndexerOptions struct with Enable field; integrated into ServerOptions struct.
pkg/server/server.go Modified NewReplicationServer function to conditionally initialize and start the indexer based on options.Indexer.Enable.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant Server
    participant Indexer

    Client->>Server: Start Replication Server
    Server->>Server: Check options.Indexer.Enable
    alt If Indexer is enabled
        Server->>Indexer: Initialize Indexer
        Indexer-->>Server: Indexer Initialized
        Server->>Indexer: Start Indexer
        Indexer-->>Server: Indexer Started
    else If Indexer is disabled
        Server->>Server: Skip Indexer Initialization
    end
    Server-->>Client: Replication Server Started
Loading

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor Author

neekolas commented Nov 25, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (2)
pkg/server/server.go (1)

104-114: Document that indexer can be nil.

Consider adding a comment to the ReplicationServer struct documenting that indx can be nil when the indexer is disabled. This helps prevent nil pointer dereferences in future code changes.

 type ReplicationServer struct {
     apiServer  *api.ApiServer
     syncServer *sync.SyncServer
     ctx          context.Context
     cancel       context.CancelFunc
     log          *zap.Logger
     registrant   *registrant.Registrant
     nodeRegistry registry.NodeRegistry
+    // indx can be nil if the indexer is disabled via options.Indexer.Enable
     indx         *indexer.Indexer
     options      config.ServerOptions
     metrics      *metrics.Server
     writerDB     *sql.DB
 }
pkg/config/options.go (1)

30-32: Consider adding a default value for clarity.

The struct follows good configuration practices with clear naming and proper tags. However, consider adding a default:"false" tag to make the default behavior explicit, consistent with other option structs in the file.

 type IndexerOptions struct {
-	Enable bool `long:"enable" env:"XMTPD_INDEXER_ENABLE" description:"Enable the indexer"`
+	Enable bool `long:"enable" env:"XMTPD_INDEXER_ENABLE" description:"Enable the indexer" default:"false"`
 }
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 1ef7cea and ad1e1f8.

📒 Files selected for processing (2)
  • pkg/config/options.go (2 hunks)
  • pkg/server/server.go (1 hunks)
🔇 Additional comments (3)
pkg/server/server.go (1)

104-114: LGTM! Verify indexer dependencies.

The conditional initialization of the indexer looks good. The error handling and initialization sequence are correct.

Let's verify if any code depends on the indexer being non-nil:

✅ Verification successful

Indexer usage is properly handled, no additional nil checks needed

The search results show that the indexer field (s.indx) is only used in two places within the server package:

  1. During initialization to start the indexer
  2. During shutdown to close the indexer

Both usages are properly guarded:

  • The start operation only occurs within the if options.Indexer.Enable block
  • The close operation is within the Close() method which already has a nil check (if s.indx != nil)

No other components directly depend on or assume the indexer's existence.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for direct usage of the indexer field
rg -A 3 "\.indx\." pkg/
# Search for any indexer-related method calls
rg -A 3 "indexer\." pkg/

Length of output: 752

pkg/config/options.go (2)

108-108: LGTM! Clean integration following established patterns.

The IndexerOptions integration into ServerOptions maintains consistent structure and naming conventions with other option groups.


30-32: Verify the implementation of the indexer toggle.

Let's ensure the new option is properly handled in the server implementation.

Also applies to: 108-108

✅ Verification successful

Indexer toggle implementation is properly handled

The verification shows that:

  • The indexer option is correctly checked in server initialization
  • Proper error handling is in place with error propagation
  • The indexer is conditionally initialized only when enabled
  • All required dependencies (writerDB, contracts, validationService) are passed to the indexer
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify proper implementation of indexer toggle

# Check the server implementation for proper handling of the indexer option
echo "Checking server implementation..."
rg -A 5 "options\.Indexer\.Enable" pkg/server/

# Check for any potential error handling related to indexer
echo "Checking error handling..."
rg -A 5 "indexer.*err" pkg/server/

Length of output: 508


Script:

#!/bin/bash
# Let's check for the complete indexer initialization and error handling context

# Get broader context around indexer initialization
echo "Checking indexer initialization context..."
rg -B 5 -A 10 "if options\.Indexer\.Enable" pkg/server/

# Check for indexer struct definition and related code
echo "Checking indexer struct and related code..."
ast-grep --pattern 'type Server struct {
  $$$
  indx $_
  $$$
}'

# Check for error handling in StartIndexer
echo "Checking StartIndexer error handling..."
rg -B 2 -A 3 "StartIndexer" pkg/server/

Length of output: 1398

@neekolas neekolas marked this pull request as ready for review November 25, 2024 21:07
@neekolas neekolas requested a review from a team as a code owner November 25, 2024 21:07
@mkysel
Copy link
Collaborator

mkysel commented Nov 25, 2024

What is the required combo of flags? Does the indexer make sense without the sync service? Can the sync service run without it? If the answer is that both must run together or neither should be run, it should be just one flag.

Copy link
Contributor Author

neekolas commented Nov 25, 2024

They can run independently, we are just likely to choose for them to be run together because it's easier to have one worker service that has a desired capacity of 1. But we could break them into two separate workers if we chose to.

Copy link
Contributor Author

neekolas commented Nov 26, 2024

Merge activity

  • Nov 25, 5:39 PM PST: A user started a stack merge that includes this pull request via Graphite.
  • Nov 25, 5:40 PM PST: A user merged this pull request with Graphite.

@neekolas neekolas merged commit e405a83 into main Nov 26, 2024
9 checks passed
@neekolas neekolas deleted the 11-25-add_option_to_manually_enable_indexer branch November 26, 2024 01:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants