Skip to content

Latest commit

 

History

History
107 lines (94 loc) · 3.03 KB

README.md

File metadata and controls

107 lines (94 loc) · 3.03 KB
Raw

cachenv

Versatile memoizing cache for program invocations, with a virtualenv-like interface.

Note: this repo is still in early development; the README is aspirational.

Overview

cachenv is a lightweight tool that provides caching for your commands, scripts, and pipelines. In fact, any program that calls exec() can use cachenv.

It's like function memoization, but at the process boundary. This is useful in a variety of contexts, especially testing and rapid iteration.

The workflow mirrors that of virtualenv - you create an environment, activate it, and work within it.

Use Cases

  • Creating consistent, dependency-free testing environments
  • Rapidly iterating on scripts or notebooks that rely on external services or data processing
  • Efficiently constructing CLI pipelines that involve large inputs or expensive filters/aggregations
  • Eliminating redundant calls to metered APIs

Usage

Initialize and activate a new cachenv:

$ cachenv init .cachenv
Created activate script at .cachenv/activate

$ source .cachenv/activate

Enable memoization for ls:

(.cachenv) $ cachenv add ls
Command 'ls' added to memoized commands.
Refreshed symlink for ls

Enjoy(?) memoization for ls:

(.cachenv) $ ls
foo

(.cachenv) $ touch bar
(.cachenv) $ ls
foo

Try diff mode:

(.cachenv) $ cachenv diff ls
0a1
> bar

Features

Implemented Feature Description
✅ Comprehensive Caching Captures stdout, stderr, and exit codes, providing a complete snapshot of a program's behavior given a particular input.
Selective Memoization Supports precise configuration to selectively enable caching based on program name, arguments, and/or input patterns.
Streaming Mode Supports caching at the line level, keyed by stdin.
File Awareness Can optionally distinguish cache entries based on the contents of files provided as arguments (e.g., for grep foo bar.txt, refresh the cache when the content of bar.txt changes).
Pipeline Compatibility Naturally integrates with command pipelines, allowing a mix of cached and live executions within complex command chains. Also includes optimizations which can effectively cache entire pipelines.
✅ Diff Mode Can show changes in a program's behavior against a cached snapshot.
✅ Cross-Environment Portability Enables cache sharing and reuse across different machines and operating systems.