Skip to content

Commit

Permalink
Add CSV logging option
Browse files Browse the repository at this point in the history
  • Loading branch information
@akhtars
akhtars committed Aug 29, 2022
1 parent 8afa2cf commit 22fcef8
Show file tree
Hide file tree
Showing 5 changed files with 65 additions and 44 deletions.
1 change: 1 addition & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ TODO
# Data files
*.xml
*.txt
*.csv
35 changes: 17 additions & 18 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ Options include:
* `-v, --verbose`: Print success status for each item
* `-d, --debug`: Save request URL and body to output log
* `-p, --prefix=<s>`: Append string to output filenames
* `-c, --csv`: Format log as CSV
* `-h, --help`: Show this message

## Examples
Expand All @@ -58,7 +59,7 @@ $ dcl-wc-metadata-api -v read numbers.txt
9000000000000: failed
908450913: read
OCLC WorldCat Metadata API: Read operation
Read 3 records, 1 failed
Read 3 records and 1 failed
Records written to wc-read-20150723112649.xml
Log written to wc-read-20150723112649-log.txt
```
Expand Down Expand Up @@ -110,7 +111,7 @@ RESULT(S)
$ dcl-wc-metadata-api create ~/Desktop/dcl-ruby/input/dwcposters-c078-marc.xml
OCLC WorldCat Metadata API: Create operation
Created 1 record, 0 failed
Created 1 record and 0 failed
Records written to wc-create-20150803155439.xml
Log written to wc-create-20150803155439-log.txt
```
Expand Down Expand Up @@ -158,7 +159,7 @@ $ dcl-wc-metadata-api -v set numbers.txt
181693: holding set
234613: holding set
OCLC WorldCat Metadata API: Set operation
Set 4 records, 1 failed
Set 4 records and 1 failed
Log written to wc-set-20220105151005-log.txt
```
Expand Down Expand Up @@ -197,7 +198,7 @@ $ dcl-wc-metadata-api -v unset numbers.txt
8096970: holding updated
8726774: update holding failed
OCLC WorldCat Metadata API: Unset operation
Unset 2 records, 1 failed
Unset 2 records and 1 failed
Log written to wc-unset-20220802152902-log.txt
```
Expand Down Expand Up @@ -228,26 +229,24 @@ RESULT(S)
```

```
$ dcl-wc-metadata-api -v check numbers.txt
$ dcl-wc-metadata-api -c -v check numbers.txt
1: 1
6567842: merged with 1
9000000000000: number check failed (Record not found.)
1,1
6567842,merged with 1
9000000000000,number check failed ("Record not found.")
OCLC WorldCat Metadata API: Check operation
Matched 1 record, 2 failed
Matched 1 record and 2 failed
Log written to wc-check-20220124175659-log.txt
Log written to wc-check-20220829154121-log.csv
```

```
# wc-check-20220124175659-log.txt
# wc-check-20220829154121-log.csv
RESULT(S)
1: 1
6567842: merged with 1
9000000000000: number check failed
{"entries"=>[{"title"=>"9000000000000", "content"=>{"requestedOclcNumber"=>"9000000000000", "currentOclcNumber"=>"9000000000000", "institution"=>"DRB", "status"=>"HTTP 404 Not Found", "detail"=>"Record not found.", "id"=>"http://worldcat.org/oclc/9000000000000", "merged"=>false, "found"=>false}, "updated"=>"2022-01-24T22:56:59.006Z"}], "extensions"=>[{"name"=>"os:totalResults", "attributes"=>{"xmlns:os"=>"http://a9.com/-/spec/opensearch/1.1/"}, "children"=>["1"]}, {"name"=>"os:startIndex", "attributes"=>{"xmlns:os"=>"http://a9.com/-/spec/opensearch/1.1/"}, "children"=>["1"]}, {"name"=>"os:itemsPerPage", "attributes"=>{"xmlns:os"=>"http://a9.com/-/spec/opensearch/1.1/"}, "children"=>["1"]}]}
Record Number,Status,Response
1,1,
6567842,merged with 1,
9000000000000,number check failed ("Record not found."),{"entries"=>[{"title"=>"9000000000000", "content"=>{"requestedOclcNumber"=>"9000000000000", "currentOclcNumber"=>"9000000000000", "institution"=>"DRB", "status"=>"HTTP 404 Not Found", "detail"=>"Record not found.", "id"=>"http://worldcat.org/oclc/9000000000000", "found"=>false, "merged"=>false}, "updated"=>"2022-08-29T19:41:21.033Z"}], "extensions"=>[{"name"=>"os:totalResults", "attributes"=>{"xmlns:os"=>"http://a9.com/-/spec/opensearch/1.1/"}, "children"=>["1"]}, {"name"=>"os:startIndex", "attributes"=>{"xmlns:os"=>"http://a9.com/-/spec/opensearch/1.1/"}, "children"=>["1"]}, {"name"=>"os:itemsPerPage", "attributes"=>{"xmlns:os"=>"http://a9.com/-/spec/opensearch/1.1/"}, "children"=>["1"]}]}
```

### Validate batch of MARCXML records
Expand All @@ -266,7 +265,7 @@ RESULT(S)
$ dcl-wc-metadata-api validate marc-batch-2018062514413340.xml
OCLC WorldCat Metadata API: Validate operation
Validated 164 records, 14 failed
Validated 164 records and 14 failed
Records written to wc-validate-20181002114734.xml
Log written to wc-validate-20181002114734-log.txt
```
Expand Down
1 change: 1 addition & 0 deletions bin/dcl-wc-metadata-api
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ EOS
opt :debug, "Save request URL and body to output log", :short => "-d"
opt :prefix, "Append string to output filenames", :type => String,
:short => "-p"
opt :csv, "Format log as CSV", :short => "-c"
stop_on SUB_COMMANDS
end

Expand Down
5 changes: 2 additions & 3 deletions dcl_wc_metadata_api.gemspec
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Gem::Specification.new do |s|
s.name = 'dcl_wc_metadata_api'
s.version = '0.3.0'
s.version = '0.4.0'
s.author = ['Shaun Akhtar']
s.email = ['[email protected]']
s.summary = 'DCL-local WorldCat Metadata API tools'
Expand All @@ -10,8 +10,7 @@ Gem::Specification.new do |s|
s.add_dependency 'json', '~> 2.0', '>= 2.0.3'
s.add_dependency 'nokogiri', '~> 1.6', '>=1.6.3'
s.add_dependency 'rest-client', '~> 2.0', '>= 2.0.1'
s.has_rdoc = false
s.date = '2018-10-02'
s.date = '2022-08-29'
s.license = 'Apache-2.0'
s.homepage = 'https://github.com/dartmouthcollegelibrary/dcl_wc_metadata_api'
s.description = "Dartmouth College Library scripts using Terry Reese's wc_metadata_api and OCLC's oclc-auth-ruby gems."
Expand Down
67 changes: 44 additions & 23 deletions lib/dcl_wc_metadata_api.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
require 'net/http'
require 'cgi'
require 'yaml'
require 'csv'

# Minor extension to evaluate API's HTTP response

Expand Down Expand Up @@ -133,7 +134,11 @@ def initialize(options={})
:debug => false
)
@debug_info = "CLIENT REQUEST(S)"
@response_status = "\nRESULT(S)\n\n"
if @global_opts[:csv]
@response_status = ["Record Number", "Status", "Response"].to_csv
else
@response_status = "RESULT(S)\n\n"
end
@response_data = Nokogiri::XML::Document.parse(
"<collection xmlns=\"http://www.loc.gov/MARC21/slim\">"
)
Expand Down Expand Up @@ -161,21 +166,22 @@ def log_output()
data.close
end

status_filename = prefix + "wc-" + @cmd + "-" + t + "-log.txt"
status_extension = @global_opts[:csv] ? ".csv" : ".txt"
status_filename = prefix + "wc-" + @cmd + "-" + t + "-log" + status_extension

# Summary
summary = ""
summary << <<~SUMMARY
OCLC WorldCat Metadata API: #{@cmd.capitalize} operation
#{PAST_TENSE[@cmd].capitalize} #{@successes.to_s} #{@successes != 1 ? "records," : "record,"} #{@failures.to_s} failed
#{PAST_TENSE[@cmd].capitalize} #{@successes.to_s} #{@successes != 1 ? "records" : "record"} and #{@failures.to_s} failed
#{"Records written to " + data_filename if any_records}
Log written to #{status_filename}
SUMMARY

# Status log
status = File.new(status_filename, "w+:UTF-8")
status.write(@debug_info) if @global_opts[:debug]
status.write(summary)
status.write(summary + "\n")
status.write(@response_status)
status.close

Expand All @@ -184,30 +190,40 @@ def log_output()

# Handle success or failure for each API call
def manage_record_result(id, result)
separator = @global_opts[:csv] ? "," : ": "
if @client.is_success?
@response_data.root << result.at_xpath(RECORD_XPATH,
"marc" => XMLNS_MARC
)
@response_status << id + ": " + PAST_TENSE[@cmd] + "\n"
puts id + ": " + PAST_TENSE[@cmd] if @global_opts[:verbose]
@response_status << id + separator + PAST_TENSE[@cmd] + "\n"
puts id + separator + PAST_TENSE[@cmd] if @global_opts[:verbose]
@successes += 1
else
@response_status << id + ": failed\n"
@response_status << result.to_s
puts id + ": failed" if @global_opts[:verbose]
if @global_opts[:csv]
@response_status << [id, "failed", result.to_s].to_csv
else
@response_status << id + separator + "failed\n"
@response_status << result.to_s
end
puts id + separator + "failed" if @global_opts[:verbose]
@failures += 1
end
end

def manage_holding_result(id, result)
separator = @global_opts[:csv] ? "," : ": "
if @client.is_success?
@response_status << id + ": holding updated\n"
puts id + ": holding updated" if @global_opts[:verbose]
@response_status << id + separator + "holding updated\n"
puts id + separator + "holding updated" if @global_opts[:verbose]
@successes += 1 if ["set", "unset"].include?(@cmd)
else
@response_status << id + ": update holding failed\n"
@response_status << result.to_s
puts id + ": update holding failed" if @global_opts[:verbose]
if @global_opts[:csv]
@response_status << [id, "update holding failed", result.to_s].to_csv
else
@response_status << id + separator + "update holding failed\n"
@response_status << result.to_s
end
puts id + separator + "update holding failed" if @global_opts[:verbose]
@failures += 1 if ["set", "unset"].include?(@cmd)
end
end
Expand All @@ -217,24 +233,29 @@ def manage_check_result(id, result)
merged = result["entries"][0]["content"]["merged"]
returnedNumber = result["entries"][0]["content"]["currentOclcNumber"]
detail = result["entries"][0]["content"]["detail"]
separator = @global_opts[:csv] ? "," : ": "

if @client.is_success? and found and returnedNumber == id
@response_status << id + ": #{returnedNumber}\n"
puts id + ": #{returnedNumber}" if @global_opts[:verbose]
@response_status << id + separator + "#{returnedNumber}\n"
puts id + separator + "#{returnedNumber}" if @global_opts[:verbose]
@successes += 1 if @cmd == "check"
else
if merged
@response_status << id + ": merged with #{returnedNumber}\n"
puts id + ": merged with #{returnedNumber}" if @global_opts[:verbose]
@response_status << id + separator + "merged with #{returnedNumber}\n"
puts id + separator + "merged with #{returnedNumber}" if @global_opts[:verbose]
else
if detail.nil?
@response_status << id + ": number check failed\n"
puts id + ": number check failed" if @global_opts[:verbose]
@response_status << id + separator + "number check failed\n"
puts id + separator + "number check failed" if @global_opts[:verbose]
else
@response_status << id + ": number check failed (#{detail})\n"
puts id + ": number check failed (#{detail})" if @global_opts[:verbose]
if @global_opts[:csv]
@response_status << [id, "number check failed (\"#{detail}\")", result.to_s].to_csv
else
@response_status << id + separator + "number check failed (\"#{detail}\")\n"
@response_status << result.to_s + "\n"
end
puts id + separator + "number check failed (\"#{detail}\")" if @global_opts[:verbose]
end
@response_status << result.to_s + "\n"
end
@failures += 1 if @cmd == "check"
end
Expand Down

0 comments on commit 22fcef8

Please sign in to comment.