Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support standard brotli compression/decompression (.br format compressible with single-threaded brotli) #351

Merged
merged 4 commits into from
Sep 6, 2023

Conversation

sebres
Copy link
Collaborator

@sebres sebres commented Sep 6, 2023

This PR implements support for standard brotli compression/decompression without 7z container, header and framing (.br format compressible with single-threaded brotli API or command line tool).

Usage examples for 7z command line:

# compress:
7z a -tbrotli -mx4 -mwlog=24 archive.txt.br file-to-compress.txt

# test (with SHA256):
7z t -scrcSHA256 archive.txt.br

# decompress:
7z e archive.txt.br -so > uncompressed-file.txt

Because brotli handler also uses BrotliEncoder/BrotliDecoder but they using brotli-mt in 7z format, also with -mmt=1, to compress/decompress single-threaded brotli format the handler uses threads=0 internally. This is artificial value and signals that standard brotli compression format (without headers and mt-frames according to RFC 7932) must be used.
However by decompression one could read the header and check whether BROTLIMT_MAGIC_SKIPPABLE is available, just brotli format is complex and it doesn't really has a header prefix, so one can't exclude that at some point some compressed file or stream wouldn't start with the same bits sequence equal to BROTLIMT_MAGIC_SKIPPABLE.
So at the moment only file extension (.br, .brotli) or parameter -tbrotli determine the type of brotli compression format.

Additionally brotli encoder extended now to support supplying of wlog and long parameters, in 7z (-m0=brotli:long=n) as well as in brotli format.

Related to #21, #126

…TLI_MAX_WINDOW_BITS (24) used by default in brotli-mt, smaller == faster);

note that :long can be set up to BROTLI_LARGE_MAX_WINDOW_BITS (30), whereas :wlog can be set up to BROTLI_MAX_WINDOW_BITS (24) only...
todo: check whether set of BROTLI_PARAM_LARGE_WINDOW to BROTLI_TRUE is needed if (lgwin > BROTLI_MAX_WINDOW_BITS)
@mcmilk mcmilk merged commit 0e8dd28 into mcmilk:master Sep 6, 2023
5 checks passed
sebres added a commit to sebres/7-Zip-zstd that referenced this pull request Sep 7, 2023
@sebres sebres deleted the st-brotli-support branch September 7, 2023 11:54
sebres added a commit to sebres/7-Zip-zstd that referenced this pull request Sep 11, 2023
…rotli codec, zip type gets missing because exceeded kNumArcsMax in registering structure g_Arcs)
sebres added a commit to sebres/7-Zip-zstd that referenced this pull request Sep 22, 2023
…rotli codec, zip type gets missing because exceeded kNumArcsMax in registering structure g_Arcs)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants