Major rewrite - support more use cases, more intuitive usage, new name

Dockerfile

@@ -1,4 +1,7 @@
-FROM golang:1.8-onbuild
-MAINTAINER Mohammed Diaa <mohammeddiaa2000@gmail.com>
+FROM golang:1.17.1-alpine as build-env
+RUN go get -v github.com/mhmdiaa/chronos
 
-ENTRYPOINT ["app"]
+FROM alpine:3.14
+RUN apk add --no-cache bind-tools ca-certificates
+COPY --from=build-env /go/bin/chronos /usr/local/bin/chronos
+ENTRYPOINT ["chronos"]

README.md

@@ -1,26 +1,98 @@
-# WaybackUnifier
+# Chronos
 
-WaybackUnifier allows you to take a look at how a file has ever looked by aggregating all versions of this file, and creating a unified version that contains every line that has ever been in it.
+Chronos (previously known as WaybackUnifier) extracts pieces of data from a web page's history. It can be used to create custom wordlists, search for secrets, find old endpoints, etc.
 
-### Installation
-Go is required.
+---
+
+## Installation
+### From binary
+Download a prebuilt binary https://github.com/mhmdiaa/chronos/releases/latest
+
+### From source
+Use `go get` to download and install the latest version
+```
+go get -u github.com/mhmdiaa/chronos
+```
+
+---
+
+## Presets
+Presets are predefined options (URL path, match regex, and extract regex) that can be used to organize and simplify different use cases. The preset definitions are stored in `~/.chronos` as JSON files
+
+```
+$ cat ~/.chronos/robots.json
+{
+        "path": "/robots.txt",
+        "match": "Disallow",
+        "extract": "(?:\\s)(/.*)"
+}
+$ chronos -pr robots -t example.com
+$ # equivalent to...
+$ chronos -p /robots.txt -m Disallow -e "(?:\\s)(/.*)"
+```
+
+---
+
+## Example usage
+
+### Extract paths from robots.txt files and build a wordlist
 ```
-go get github.com/mhmdiaa/waybackunifier
+$ chronos -t example.com -p /robots.txt -m Disallow -e "(?:\\s)(/.*)" -o robots_wordlist.txt
 ```
-This will download the code, compile it, and leave a `waybackunifier` binary in $GOPATH/bin.
 
-### Syntax
+### Save all versions of a web page locally and filter out a specifc status code
+```
+$ chronos -t http://example.com/this_is_403_now_but_has_it_always_been_like_this_question_mark -fs 403 -od output
+```
+
+### Save URLs of all subdomains of example.com that were last seen in 2015
+```
+$ chronos -t *.example.com -u -to 2015
+```
+
+### Run the S3 preset that extract AWS S3 URLs
+```
+$ chronos -pr s3 -t example.com
+```
+
+
+---
+
+## Options
 ```
-  -concurrency int
-        Number of requests to make in parallel (default 1)
-  -output string
-        File to save results in (default "output.txt")
-  -sub string
-        list of comma-separated substrings to look for in snapshots (snapshots will only be considered if they contnain one of them) (default "Disallow,disallow")
-  -url string
-        URL to unify versions of (without protocol prefix) (default "site.com/robots.txt")
+Usage: chronos <preset (optional)> <params>
+  -c int
+    	Number of concurrent threads (default 10)
+  -e string
+    	Extract regex
+  -fm string
+    	Filter Mime codes
+  -from string
+    	Match results after a specific date (Format: yyyyMMddhhmmss)
+  -fs string
+    	Filter status codes
+  -m string
+    	Match regex
+  -mm string
+    	Match Mime codes
+  -ms string
+    	Match status codes (default "200")
+  -o string
+    	Output file path (default "output.txt")
+  -od string
+    	Directory path to store matched results' entire pages
+  -p string
+    	Path to add to the URL
+  -preset string
+    	Preset name
+  -t string
+    	Target URL/domain (supports wildcards)
+  -to string
+    	Match results before a specific date (Format: yyyyMMddhhmmss)
+  -u	URLs only
 ```
 
-The settings are by default suitable for unifying robots.txt files. Feel free to change the value of `-sub` to anything else, or supply an empty string to get all versions of a file without filtering.
+---
 
-**Note:** Lines are saved *unordered* for performance reasons
+## Contributing
+Find a bug? Got a feature request? Have an interesting preset in mind? Issues and pull requests are always welcome :)

examples/robots.json

@@ -0,0 +1,5 @@
+{
+        "path": "/robots.txt",
+        "match": "Disallow",
+        "extract": "(?:\\s)(/.*)"
+}

examples/s3.json

@@ -0,0 +1,3 @@
+{
+        "extract": "(?i)^([https:\\/\\/]*s3\\.amazonaws.com[\\/]+.*|[a-zA-Z0-9_-]*\\.s3\\.amazonaws.com\\/.*)$"
+}

go.mod

@@ -0,0 +1,3 @@
+module github.com/mhmdiaa/chronos
+
+go 1.16