Skip to content

Commit

Permalink
Implement Sentry integration
Browse files Browse the repository at this point in the history
This patch adds support for Workers to emit various RSpecQ internal events to
a Sentry[1] instance.

Reporting to Sentry is opt-in and is enabled by setting the SENTRY_DSN
environment variable (see
https://github.com/getsentry/raven-ruby#raven-only-runs-when-sentry_dsn-is-set).
If SENTRY_DSN is not set this is a noop.

Events emitted:

- [error] the spec file split command failed; slow spec files files will
  be scheduled as a whole so build times will be negatively impacted
- [warning] no previous timings were found in Redis; scheduling will be random
  so build times will be negatively impacted
- [info] slow files were detected and will be split, including the
  actual filenames

The above are still printed to the standard output of the worker.

Closes #8

[1] https://sentry.io
  • Loading branch information
agis committed Aug 8, 2020
1 parent 83bd0c0 commit 3364dd0
Show file tree
Hide file tree
Showing 4 changed files with 46 additions and 14 deletions.
2 changes: 1 addition & 1 deletion bin/rspecq
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ OptionParser.new do |o|
BANNER

o.separator ""
o.separator "OPTIONS:"
o.separator "OPTIONS:"

o.on("-b", "--build ID", "A unique identifier for the build. Should be " \
"common among workers participating in the same build.") do |v|
Expand Down
1 change: 1 addition & 0 deletions lib/rspecq.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
require "rspec/core"
require "sentry-raven"

module RSpecQ
# If a worker haven't executed an example for more than WORKER_LIVENESS_SEC
Expand Down
56 changes: 43 additions & 13 deletions lib/rspecq/worker.rb
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ class Worker

# Retry failed examples up to N times (with N being the supplied value)
# before considering them legit failures
#
#
# Defaults to 3
attr_accessor :max_requeues

Expand Down Expand Up @@ -102,10 +102,11 @@ def try_publish_queue!(queue)

timings = queue.timings
if timings.empty?
# TODO: should be a warning reported somewhere (Sentry?)
q_size = queue.publish(files_to_run.shuffle)
puts "WARNING: No timings found! Published queue in " \
"random order (size=#{q_size})"
log_event(
"No timings found! Published queue in random order (size=#{q_size})",
"warning"
)
return
end

Expand All @@ -114,7 +115,12 @@ def try_publish_queue!(queue)
end.map(&:first) & files_to_run

if slow_files.any?
puts "Slow files (threshold=#{file_split_threshold}): #{slow_files}"
log_event(
"Slow files detected (threshold=#{file_split_threshold}): #{slow_files}",
"info",
slow_files: slow_files,
slow_files_count: slow_files.count,
)
end

# prepare jobs to run
Expand Down Expand Up @@ -168,14 +174,22 @@ def files_to_example_ids(files)
out = `#{cmd}`

if !$?.success?
# TODO: emit warning to Sentry
puts "WARNING: Error splitting slow files; falling back to regular scheduling:"

begin
pp JSON.parse(out)
rescue JSON::ParserError
puts out
end
puts out
puts $?.inspect
rspec_output = begin
JSON.parse(out)
rescue JSON::ParserError
out
end

log_event(
"Failed to split slow files, falling back to regular scheduling",
"error",
rspec_output: rspec_output,
cmd_result: $?.inspect,
)

pp rspec_output
puts

return files
Expand All @@ -192,5 +206,21 @@ def relative_path(job)
def elapsed(since)
Process.clock_gettime(Process::CLOCK_MONOTONIC) - since
end

def log_event(msg, level, additional={})
puts msg

Raven.capture_message(msg, level: level, extra: {
build: @build_id,
worker: @worker_id,
queue: queue.inspect,
files_or_dirs_to_run: @files_or_dirs_to_run,
populate_timings: populate_timings,
file_split_threshold: file_split_threshold,
heartbeat_updated_at: @heartbeat_updated_at,
object: self.inspect,
pid: Process.pid,
}.merge(additional))
end
end
end
1 change: 1 addition & 0 deletions rspecq.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Gem::Specification.new do |s|
end

s.add_dependency "redis"
s.add_dependency "sentry-raven"

s.add_development_dependency "rake"
s.add_development_dependency "pry-byebug"
Expand Down

0 comments on commit 3364dd0

Please sign in to comment.