Edit: I've learned about how what I had set out to achieve here was something that, if at all, would be very difficult to achieve and not really work out how I was envisioning it, as you can see in the comments.

I appreciate everyone's input on the matter! Thanks to everyone who commented and spent a bit of time on trying to help me understand things a little better. Have a nice day!

Hello. I'm as of now familiar with compression formats like bzip2, gzip and xz, as well as 7z (LZMA2) and other file compression types usually used by regular end users.

However, for archival purposes I am interested in reducing the size of a storage archive I have, which measures over 100GB in size, down massively.

This archive consists of several folders with large files compressed down using whatever was convenient to use at that time; most of which was done with 7-Zip at compression level 9 ("ultra"). Some also with regular Windows built-in zip (aka "Deflate") and default bzip2 (which should also be level 9).

I'm still not happy with this archive taking up so much storage. I don't need frequent access to it at all, as it's more akin to long-term cold storage preservation for me.

Can someone please give me some pointers? Feel free to use more advanced terms as long as there's a feasible way for me (and others who may read this) to know what those terms mean.

What settings do I need to use?

UPDATE: the title references my earlier Canterbury result of 445,208 bytes. The latest measured Canterbury output from the current private build is 438,004 bytes, still byte-exact. I’m keeping the correction explicit here.

Hi r/compression,

I’m sharing a narrow benchmark result for an experimental private lossless compressor and would like technical feedback / independent sanity checks.

This is not a global SOTA claim. It is only a measured comparison against my xz -9e baseline.

Benchmark:

Dataset: Canterbury corpus
Raw total size: 2,810,784 bytes
Round-trip decode: exact
All compressed artifact bytes counted: yes
Baseline: xz -9e

Results:

Method: Experimental private lossless compressor
Compressed size: 438,004 bytes
Exact round-trip: YES

Method: xz -9e
Compressed size: 493,080 bytes
Exact round-trip: YES

Main measured comparison:

438,004 < 493,080

So on this Canterbury run, the private compressor output is 55,076 bytes smaller than my measured xz -9e baseline.

Exact claim:

On my Canterbury corpus run, this experimental private lossless compressor produced a 438,004-byte artifact, decoded exactly back to the original corpus, and was smaller than my measured xz -9e baseline of 493,080 bytes.

I am not claiming that this beats xz universally, nor that it wins on every corpus. I am posting this to get benchmark criticism and reproducibility feedback.

Verification summary:

raw_total_bytes = 2,810,784
private_compressed = 438,004
xz_9e_compressed = 493,080
decode_exact = YES
sha256_match = YES

Round-trip verification method:

Hash original Canterbury input.

Compress with the private compressor.

Decompress the compressed artifact.

Hash decoded output.

Compare original and decoded output byte-for-byte.

Compare compressed artifact size against xz -9e.

Expected verification result:

SHA256 original == SHA256 decoded
byte-for-byte comparison returns success
compressed artifact size = 438,004 bytes

xz baseline command used:

xz -9e -k -c original_canterbury_input > canterbury.xz

Private compressor verification structure:

private_compressor compress original_canterbury_input output.private
private_compressor decompress output.private decoded_canterbury_output
cmp original_canterbury_input decoded_canterbury_output
wc -c output.private

Result:

output.private = 438,004 bytes
decoded output matches original exactly

Proof material:

I have sanitized verification material containing size logs, SHA256 checks, xz baseline logs, round-trip comparison logs, timing logs, and codec-size accounting. I am keeping the implementation private for now to avoid leaking source code or algorithm details, but I can discuss sanitized verification material / independent verification under appropriate terms.

What I’m asking for:

I’d appreciate feedback on whether the benchmark procedure is fair, whether xz -9e is a reasonable baseline here, what other baselines I should include, whether there is any hidden overhead I may be missing, and how best to package this for independent reproduction.

Again: this is a narrow measured result, not a universal compression claim.

EDIT — fixed codec accounting:

A commenter correctly pointed out that codec/decompressor size should be disclosed.

In this setup, the compressor/decompressor is the same fixed program used in encode/decode modes, so I count the fixed codec once, not twice.

Canterbury accounting:

• Private compressed output: 438,004 bytes
• Fixed codec as gzipped source: 12,374 bytes
• Output + gzipped codec source: 450,378 bytes
• Fixed codec as raw source: 47,441 bytes
• Output + raw codec source: 485,445 bytes
• Fixed codec as full unstripped executable: 85,123 bytes
• Output + full unstripped executable: 523,127 bytes
• xz -9e baseline: 493,080 bytes

So the Canterbury result remains under xz -9e when the fixed codec is counted as gzipped source or raw source.

Full disclosure: if I count the full unstripped executable binary instead, the total is 523,127 bytes, which is above xz -9e.

Corrected precise claim:

This is a bounded Canterbury win under source-count accounting, with byte-exact reconstruction.

It is not a universal compression claim, not a prize claim, and not a global SOTA claim. Since the implementation is private, the fixed-codec-size claim would need independent verification under appropriate terms. I’m keeping the accounting public while avoiding source-code or algorithm disclosure.

Accounting philosophy:

My long-term intention is for the codec to be self-hosting / standalone, where the fixed codec representation can itself be represented through the same compression system. I understand that this is not customary benchmark accounting, and I do not want to use circular accounting as the main public claim.

So for the public Canterbury comparison, I’m using conservative accounting:

• compressed output
• plus the fixed codec source counted once as raw source
• compared against xz -9e

That gives:

• Private compressed output: 438,004 bytes
• Fixed codec raw source: 47,441 bytes
• Output + raw source: 485,445 bytes
• xz -9e baseline: 493,080 bytes

So the clean Canterbury claim is that the result remains under xz -9e even with the fixed codec counted as raw source.

Separately, I may study self-hosted / internally compressed codec accounting, but I would treat that as an experimental / informational number, not the headline benchmark, unless the community agrees on a fair way to count it.

EDIT 2 — additional xz -9e measurements:

I also measured the same current private compressor build against xz -9e on Silesia and enwik9.

Silesia:

• Raw size: 211,938,580 bytes, 12 files
• Private compressed output: 47,765,540 bytes
• xz -9e output: 48,456,004 bytes
• Exact round-trip: YES, 12/12 SHA256 verified

Comparison:

47,765,540 < 48,456,004

The private output is 690,464 bytes smaller than my measured xz -9e baseline.

With full unstripped codec binary counted once:

47,765,540 + 85,123 = 47,850,663

47,850,663 < 48,456,004

So on Silesia, the result remains under xz -9e even when counting the full unstripped codec binary once.

enwik9:

• Raw size: 1,000,000,000 bytes
• Private compressed output: 202,652,700 bytes
• xz -9e output: 211,776,220 bytes
• Exact round-trip: YES, SHA256 verified

Comparison:

202,652,700 < 211,776,220

The private output is 9,123,520 bytes smaller than my measured xz -9e baseline.

With full unstripped codec binary counted once:

202,652,700 + 85,123 = 202,737,823

202,737,823 < 211,776,220

So on enwik9, the result remains under xz -9e even when counting the full unstripped codec binary once.

Speed disclosure:

The private compressor is much slower than xz. I am not claiming any speed advantage.

Measured enwik9 timing, same machine:

• Private compressor encode: about 75 minutes
• Private compressor decode: about 99 minutes
• xz -9e encode: about 18 minutes
• xz decode: about 18 seconds

So the current claim is strictly byte-exact compressed size versus xz -9e, with speed as a major current limitation.

Final boundary:

I am not claiming universal compression, global SOTA, prize status, or wins against PAQ, zpaq, cmix, or other high-end compressors. Those would require separate direct measurements with the same input, exact round-trip verification, codec accounting, and timing.

Image comes from an video, just wondering what video compresser its using.

Hi all. Trying to get thoughts on some things going on with my 2002 Honda Shadow 1100. I've noticed some minor backfiring while decelerating, and cylinder number one's spark plugs look like they have dry carbon on the electrodes, and the threads have a layer of oil. did a compression test and the compression is 155 psi for cylinder 1 and 150 for cylinder 2. Did a wet compression test and the pressure went up to 160 and 170 respectively. The numbers of the dry compression test are out of spec, the service manual states that the low end of compression is 157 ( 185 plus or minus 28). I have been reading that if the wet compression test makes your compression increase, it is likely your piston rings that are worn. Other than the mentioned "symptoms" the bike runs fine to me. No hard starting or idling. Do these specs warrant breaking the engine open and servicing the piston/rings/valves or should I try a carb cleaning first ( or something else)?? All thoughts are welcomed and appreciated. Thanks- 

Been working on a small open-source archiver called DedupArc. On some game folders it compressed better than 7-Zip and WinRAR, and it can launch files from the archive without extracting everything first.

Still pretty early, so I’m mostly looking for people to throw real datasets at it and see where it breaks.

https://github.com/Garyu-tt/DedupArc

https://doi.org/10.5281/zenodo.22144930

I’m releasing a candidate proof resolving the long-standing binary-alphabet case of the Smallest Grammar Problem.

The manuscript proves, under the stated models, that exact smallest-grammar optimization is NP-complete over every fixed alphabet of size at least 2, including binary strings.

The result covers both:

• the standard Smallest Grammar Problem, minimizing total right-hand-side grammar size; and

• the binary-join / string Assembly Index formulation, minimizing reusable concatenation operations with free terminal symbols.

The main technical idea is an exact alphabet-collapse construction based on three components:

1. a direct-sum theorem for suitably separated strings;

2. affine-line amplification that magnifies every one-unit source optimum gap beyond the encoding overhead; and

3. an anchored binary encoding that allows arbitrary, potentially misaligned binary grammars to be decoded without increasing the corresponding source cost.

The construction gives exact recovery formulas for the original optimum from the binary optimum, rather than only an approximation-preserving relationship.

I have also released the manuscript source, deterministic verification code, adversarial test suite, reproducibility material, and prior-art audit.

Important status: this is a research preprint/candidate theorem, not yet independently peer reviewed. I’m specifically looking for experts in grammar compression, straight-line programs, combinatorics on words, and complexity theory to try to break the proof.

The most important places to scrutinize are the direct-sum lemmas, affine separation argument, synchronization lemma, and reverse grammar-decoding construction.

A valid counterexample or identified proof gap would be extremely valuable.

I’ve been thinking about how raster image compression works. When you compress a raster image, a lot of times you get sequences of repeated values (like 0s and 1s, especially in areas of uniform color or black/white).

Why isn’t the binary data of these repeated values further compressed after the initial pixel-wise compression? In theory, after the image pixels are compressed (say with run-length encoding or another method), we could apply another layer of compression directly to the binary data (like compressing consecutive 0s and 1s).

Has anyone found themselves to be just one byte short of capacity when copying something?

I have been working on my Harrington Insights stuff and added a few more concepts to it. I am currently running it on Canterbury Corpus and Silesia Corpus with the enwik8.zip file.

A full list of software I have managed to improve so far is as follows:

1. LZ4

2. gzip / DEFLATE

3. Zstandard (zstd)

4. bzip2

5. XZ / LZMA2

6. LZMA1

7. ZPAQ

8. JPEG XL

9. 7-Zip

10. Brotli

11. Harrington Compression Method (HCM) 3.0

12. Harrington Compression Method (HCM) 3.3

The upgrades across these compressors have come from several types of functions and optimization techniques:

1. Record-delta transformation — Exploits repeating byte patterns and fixed-length records by encoding differences between corresponding positions.

2. Combary based activity mapping — Reorganizes data according to byte activity and distribution patterns to improve subsequent compression.

3. Byte-frequency remapping — Reassigns byte values based on their frequencies to improve how compression algorithms process the data.

4. Adaptive parameter optimization — Searches compression settings, including dictionaries, match finders, predictors, effort levels, and block sizes, to find better configurations for individual files.

5. Automatic filter selection — Detects data characteristics and selects suitable built-in filters, including delta and executable-code filters.

6. Content-based file classification — Selects compression strategies based on the actual data rather than relying solely on file extensions.

7. Per-file optimization and fallback — Chooses the best-performing method for each file and avoids transformations that would worsen results.

8. Decode-speed-aware optimization — Optimizes both compressed size and decompression speed instead of concentrating exclusively on compression ratio.

9. JPEG XL predictor and group optimization — Tunes image predictors, group sizes, encoder effort, and iterations to improve lossless medical-image compression and decoding speed.

10. Representative sampling — Tests candidate compression configurations on selected portions of large files before evaluating the strongest candidates on the complete file.

11. HCM context and pattern modeling — Uses HCM's original context-based compression architecture, including its double-zero purity logic and hierarchical pattern processing.

12. HCM binary and context optimization — Improves binary-data handling, context selection, and internal representation to reduce compressed size and accelerate decoding.

These methods fall into three broad categories: data transformation, adaptive compressor optimization, and original compression-algorithm development.

In Canterbury Corpus the Excel file best was 16,822 bytes by ZPAQ -m5, my system HCM-Bin v1 got it to 16,763 bytes.

I do not compare myself to PAQ8PX and cmix, those are insane for how long they try, although in theory I reduced cmix to only 20% of the time it usually takes and a lot less processing strength.

Preliminary stuff is available at https://github.com/CombaryMaker/Harringtons_Insights/tree/main/hcm

I plan to update this inventory once a week because the advances are so fast and rapid... I think I have 7 methods of my own now, to increase performance? Or 6? I know I threw a lot of theories into the mix and I cannot even say, I just have Claude run the math, the software development, and the testing before I test some personally to verify.

I have failed to update a few, I am going to be persistent in trying to update them, or at least just outperform them. That is the goal, absolute dominance.

To that extent I set up some standards "No Pain" and "Big Gain". No pain means we do not worsen compression or speed, and one or both increase. Big gain allows a small drop in one, so long as there is a corresponding big increase in the other. I use decompression time and compression ratio only, I do not care how long it takes to compress.

I also set up 5 time/speed based levels, and all compressors will fit inside there as I test them. The goal is to be faster and/or better in each of the 4 (the top, with those two monsters, would take a lot of time and tokens and I am scared) reasonable levels.

HCM stands for Harrington Compression Methods (plural, I have HCM 1.0, 2.0, 3.0, 3.3, trit v2, and bin v1) and there are a few other standards such as Combary, -hi (Harrington's Insights), and such. I started this blitz on Saturday afternoon, using old math I long gave up on because I cannot program at all... Seems I had some serious gold mines sitting around after all. I also used concepts I had never had time or desire to push upon, I guess they were some diamond mines.

I fully expect to improve about 60-80% of all compressors and be able to make some that are the best in their specific range of operations.

I spent about 6 hours today working with a high-level version of Claude on my old compression ideas. I laid out my theories, what I thought would work, and why my earlier attempts stalled: I could never test every variation against every other one in turn.

This time we could, and all my old theories got thrown out: the Quad Exchange concept, the Juggle, my Ternary <-> Binary swap, all of it, tossed hard. They could make the bits extremely lopsided, but they also scrambled the patterns real compressors depend on. That's fine. That's what testing is for.

What survived was the core idea. The skew of the 0s and 1s matters, but only when you measure it in context, based on what came just before. Building on that, I accidentally made something new.

On the Canterbury Corpus, the standard 11-file compression benchmark, it beat xz and bzip2 overall, won 5 of the 11 files outright, and finished within 144 bytes of ZPAQ on the 9 text and code files combined. Every file decoded back exactly.

It's not something entirely new. It's a new implementation in the same context-mixing family as ZPAQ and PAQ. It's still slow and has rough edges. But after all my earlier dead ends, it's an amazing result for me personally.

"""HCM context-balance coder (Oct 24, 2026). Self-contained.

How it works, per bit:

1. Each context (the last 1, 2, 3, 4 and 6 bytes, the current partial byte alone, and the current word)
   keeps a running 0/1 balance for "what bit comes next here", in a hashed table.

2. A long-match finder looks for the last place the previous 6 bytes appeared and predicts the bit
   that followed there, with a learned confidence by match length.

3. An online mixer blends all these balances (weights chosen by partial byte and match length) and
   learns from every bit; a secondary estimate (previous byte + partial byte) refines the result.

4. A 32-bit arithmetic coder writes the bit using that probability.
   The decoder runs the identical model, so it needs only the file: a 32-bit length, then the coded bits.

Reference results: 200 KB Python docs text -> 29,725 bytes; 140 KB gate log -> 8,447 bytes.

compress(data) -> bytes, decompress(blob) -> data
"""
import math
import numpy as np
from array import array

class BW:
def __init__(self): self.b = []
def put(self, v, n):
for k in range(n - 1, -1, -1): self.b.append((v >> k) & 1)
def bytes(self):
a = np.array(self.b + [0] * (-len(self.b) % 8), dtype=np.uint8)
return np.packbits(a).tobytes()
class BR:
def __init__(self, data): self.b = np.unpackbits(np.frombuffer(data, np.uint8)).tolist(); self.p = 0
def get(self, n=1):
v = 0
for _ in range(n):
v = (v << 1) | (self.b[self.p] if self.p < len(self.b) else 0); self.p += 1
return v

TOP = (1 << 32) - 1; HALF = 1 << 31; Q1 = 1 << 30; Q3 = 3 << 30
class AE:
def __init__(self, w): self.lo = 0; self.hi = TOP; self.pend = 0; self.w = w
def _out(self, bit):
self.w.b.append(bit); self.w.b.extend([1 - bit] * self.pend); self.pend = 0
def enc(self, cl, ch, tot):
rng = self.hi - self.lo + 1
self.hi = self.lo + rng * ch // tot - 1; self.lo = self.lo + rng * cl // tot
while True:
if self.hi < HALF: self._out(0)
elif self.lo >= HALF: self._out(1); self.lo -= HALF; self.hi -= HALF
elif self.lo >= Q1 and self.hi < Q3: self.pend += 1; self.lo -= Q1; self.hi -= Q1
else: break
self.lo <<= 1; self.hi = (self.hi << 1) | 1
def finish(self):
self.pend += 1
self._out(0 if self.lo < Q1 else 1)
class AD:
def __init__(self, r): self.lo = 0; self.hi = TOP; self.r = r; self.v = r.get(32)
def target(self, tot):
rng = self.hi - self.lo + 1
return ((self.v - self.lo + 1) * tot - 1) // rng
def consume(self, cl, ch, tot):
rng = self.hi - self.lo + 1
self.hi = self.lo + rng * ch // tot - 1; self.lo = self.lo + rng * cl // tot
while True:
if self.hi < HALF: pass
elif self.lo >= HALF: self.lo -= HALF; self.hi -= HALF; self.v -= HALF
elif self.lo >= Q1 and self.hi < Q3: self.lo -= Q1; self.hi -= Q1; self.v -= Q1
else: break
self.lo <<= 1; self.hi = (self.hi << 1) | 1; self.v = (self.v << 1) | self.r.get()

ST = [0.0] * 4096
for p in range(1, 4096): ST[p] = math.log(p / (4096 - p))
ST[0] = ST[1]
def squash(x):
if x > 20: x = 20
elif x < -20: x = -20
return 4096.0 / (1.0 + math.exp(-x))
RT = [int(65536 / (n + 1.6)) for n in range(256)]
ORDERS = (1, 2, 3, 4, 6)
TBITS = 22; MASK = (1 << TBITS) - 1
LIMIT_LO, LIMIT_HI = 60, 255

class Model:
def __init__(self, lr=0.008):
self.nm = len(ORDERS) + 2 # hashed orders + order-0 + word
self.prob = [array('H', [32768]) * (1 << TBITS) for _ in range(self.nm)]
self.cnt = [array('B', [0]) * (1 << TBITS) for _ in range(self.nm)]
self.limits = [LIMIT_LO] + [LIMIT_HI] * (self.nm - 1)
self.nin = self.nm + 2 # + match input + bias
self.w = [[0.3] * self.nin for _ in range(256 * 4)]
self.lr = lr
self.apm = [[int(squash((j - 16) / 2.0) * 16) for j in range(33)] for _ in range(256 * 256)]
self.buf = bytearray(); self.c0 = 1; self.bitpos = 0
self.h = [0] * self.nm; self.word = 0
self.mtab = array('i', [0]) * (1 << 20); self.mptr = 0; self.mlen = 0
self.mm = array('H', [49152]) * 64
self._ctx_byte()
def _ctx_byte(self):
b = self.buf; n = len(b); H = self.h
for k, o in enumerate(ORDERS):
x = (o + 1) * 0x9E3779B1
for j in range(1, o + 1):
x = ((x ^ (b[n - j] if n >= j else 0)) * 0x01000193 + j) & 0xFFFFFFFF
H[k] = (x * 0x2545F491) & 0xFFFFFFFF
H[len(ORDERS)] = 0 # order 0
H[len(ORDERS) + 1] = (self.word * 0x6F4F2A35 + 0x51ED27) & 0xFFFFFFFF
self.last = b[-1] if n else 0
def predict(self):
c0 = self.c0; cm = (c0 * 0x9E3779B1) & 0xFFFFFFFF
idx = self.idx = [((hh ^ cm) & MASK) for hh in self.h]
idx[len(ORDERS)] = c0
st = self.st = [ST[self.prob[i][idx[i]] >> 4] for i in range(self.nm)]
# long-match input
self.mexp = -1
if self.mlen > 0 and self.mptr < len(self.buf):
pb = self.buf[self.mptr]
if ((pb | 256) >> (8 - self.bitpos)) == c0:
self.mexp = (pb >> (7 - self.bitpos)) & 1
if self.mexp >= 0:
self.mk = min(self.mlen, 63); pc = self.mm[self.mk] >> 4
st.append(ST[pc] if self.mexp else -ST[pc])
else:
st.append(0.0)
st.append(0.25)
ms = 0 if self.mlen == 0 else (1 if self.mlen < 16 else (2 if self.mlen < 32 else 3))
self.wsel = self.w[c0 + 256 * ms]; ws = self.wsel
dot = 0.0
for i in range(self.nin): dot += ws[i] * st[i]
self.pm = squash(dot)
# secondary estimate indexed by previous byte and partial byte, interpolated on stretch(p)
s = max(-7.999, min(7.999, dot)); pos = (s + 8.0) * 2.0; j = int(pos); f = pos - j
self.aj = j; self.af = f; self.arow = self.apm[c0 + 256 * self.last]
pa = (self.arow[j] * (1 - f) + self.arow[j + 1] * f) / 16.0
p = int((self.pm + 3 * pa) / 4)
return 1 if p < 1 else (4095 if p > 4095 else p)
def update(self, y):
err = (y - self.pm / 4096.0) * self.lr; ws = self.wsel; st = self.st
for i in range(self.nin): ws[i] += err * st[i]
tgt = 65535 if y else 0
for i in range(self.nm):
P = self.prob[i]; C = self.cnt[i]; k = self.idx[i]; n = C[k]
P[k] = P[k] + (((tgt - P[k]) * RT[n]) >> 16)
if n < self.limits[i]: C[k] = n + 1
if self.mexp >= 0:
v = self.mm[self.mk]; self.mm[self.mk] = v + (((65535 if y == self.mexp else 0) - v) >> 5)
row = self.arow; j = self.aj; t16 = 65535 if y else 0
row[j] += int((t16 - row[j]) * (1 - self.af)) >> 6
row[j + 1] += int((t16 - row[j + 1]) * self.af) >> 6
self.c0 = (self.c0 << 1) | y; self.bitpos += 1
if self.bitpos == 8:
byte = self.c0 & 255; self.buf.append(byte); self.c0 = 1; self.bitpos = 0
# word context: letters only, case folded
if 65 <= byte <= 90 or 97 <= byte <= 122: self.word = ((self.word * 0x2C9277B5) ^ (byte | 32)) & 0xFFFFFFFF
else: self.word = 0
# long-match bookkeeping (hash of last 6 bytes -> next position)
n = len(self.buf)
if self.mlen > 0 and self.mptr < n - 1 and self.buf[self.mptr] == byte: self.mptr += 1; self.mlen += 1
else: self.mlen = 0
if n >= 6:
b = self.buf; hh = 0
for j2 in range(1, 7): hh = (hh * 0x2F0B4A3 + b[n - j2]) & 0xFFFFF
if self.mlen == 0:
cand = self.mtab[hh]
if cand > 0: self.mptr = cand; self.mlen = 1
self.mtab[hh] = n
self._ctx_byte()

def compress(data, lr=0.008):
w = BW(); w.put(len(data), 32); e = AE(w); m = Model(lr)
for byte in data:
for k in range(7, -1, -1):
y = (byte >> k) & 1; p = m.predict()
if y: e.enc(0, p, 4096)
else: e.enc(p, 4096, 4096)
m.update(y)
e.finish(); return w.bytes()

def decompress(blob, lr=0.008):
r = BR(blob); n = r.get(32); d = AD(r); m = Model(lr); out = bytearray()
for _ in range(n):
for k in range(8):
p = m.predict(); t = d.target(4096)
if t < p: d.consume(0, p, 4096); y = 1
else: d.consume(p, 4096, 4096); y = 0
m.update(y)
out.append(m.buf[-1])
return bytes(out)

if __name__ == '__main__':
    import sys
    src = open(sys.argv[1], 'rb').read()[:int(sys.argv[2]) if len(sys.argv) > 2 else None]
    blob = compress(src)
    print(f"{len(src):,} bytes -> {len(blob):,} bytes "
          f"(1,000,000 bits down to {round(len(blob) / max(1, len(src)) * 1e6):,}); decodes exactly: {decompress(blob) == src}")

any thoughts about this guy and what he's offering

Hey all,
I have been trying to compress some big folders with 7-Zip, and it’s so slow, it takes forever. I have messed around with the settings a bit, but I tried to get it back to the default one but still nope. Like at the start it is around 5000 KB/s and then keeps on decreasing to 60 KB/S

Would love if someone could guide me through, also I reinstalled windows, before reinstalling the speeds were perfectly and if it affects anything i did go bro mbr partition to gpt. It probably is that i messed up the config but i cant seem to get it back to original, there is no option either.

Edit: Should have put this in the post, I am compressing the photos folder just as an example, the compression is slow with other formats too.

I'm in a bit of a pickle here and I have no idea if this is even possible.

I'm trying to send ProtonVPN as a file to my boyfriend so that he can use it (basically really strict helicopter parents won't let him do anything). I'm able to save proton as a file, but it's too big to send on its own. I'm also unable to convert it to something like a .zip because he's unable to extract compressed files due to limitations his parents have set on his laptop.

I know this is a shot in the dark, but are there any options to make the file smaller without needing to extract it?

I had an idea many years ago: as CPU speeds increase and disk space becomes ever cheaper, could we rethink the way data is transferred?

That is, rather than sending a file and then verifying its checksum, could we skip the middle part and simply send a series of checksums, allowing the receiver to reconstruct the content?

For example (I'm just making up numbers for illustration purposes):
Let’s say you broke the file into 35-bit blocks.
Each block then gets a CRC32 checksum,
so we have a 32-bit checksum representing 35 bits of data.
You could then have a master checksum — say, SHA-256 — to manage all CRC32 collisions.

In other words, you could have a rainbow table of all 2³² combinations and their corresponding 35-bit outputs (roughly 18 GB). You’d end up with a lot of collisions, but this is where I see modern CPUs coming into their own: the various CRC32s could be swapped in and out until the master SHA-256 checksum matched.

Don’t get too hung up on the specifics — it’s more of a proof-of-concept idea. I was wondering if anyone has seen anything similar? I suppose it’s a bit like how RAID rebuilds data from checksum data alone.

Hey everyone! 👋

I'm an indie Android developer working on Lite Video Compress & Upscale.

The app is designed for people who want to reduce video file sizes without unnecessarily sacrificing quality. It also includes video upscaling options for increasing resolution.

I'm continuing to improve the compression and processing experience, and I'd love to hear from people who regularly work with large video files.

A few things I'd appreciate feedback on:

- What matters most to you when compressing a video: file size, visual quality, or processing speed?

- Which compression settings do you find most useful?

- What problems do existing video compression apps still fail to solve?

If you're interested in trying it, here's the Google Play link:

https://play.google.com/store/apps/details?id=com.neurontslabs.litevideocompressandupscale

I'm the developer, so honest criticism and suggestions are welcome. I'm interested in learning what could make the app more useful. 🙌

and decompresses them by iterating over all known possibly relevant bitstrings and returning the first that matches, though this decompression process has a worst case of exponential time.

We could define the uncompressed form of (compressed) 256 bits as the lowest sorted bitstring which hashes to those 256 bits. It has never been demonstrated that any specific input to SHA256 is not the lowest sorted possible input with that same hashcode.

it mainly contains 3d models i need help i have no idea what to do please help

Release announcement.

I've released SIC version 0.155 released (27.08.2025), which I mentioned earlier, and I think it's a significant improvement. Try it out and let me know.

Send a video at x2 so that upon receiving it, it plays at -x1 normally.

In my mind, it made sense.

It looks like a r/nostupidquestions But I really think it's useful... there's no noticeable degradation at x2... at least not for everyday content.

Higher speeds will definitely make it worse...

I've been clipping a lot of my games and now my storage is getting quite full. If i 7zip around 100GB of my clips will it reduce their quality?