220 Commits

Author	SHA1	Message	Date
Emmanuel Marty	ce7fc33646	Reduce memory use	2019-11-12 00:30:24 +01:00
Emmanuel Marty	53b2013b73	Small improvement to merging large matches	2019-11-11 18:41:08 +01:00
Emmanuel Marty	7371486513	Bump version	2019-10-29 12:10:13 +01:00
Emmanuel Marty	f249597dfd	Increase LZSA2 ratio	2019-10-29 12:09:14 +01:00
Emmanuel Marty	78f588a833	Clarify defines	2019-10-29 10:45:57 +01:00
Emmanuel Marty	3c4f535e0b	Increase LZSA2 ratio by ~0.02%	2019-10-27 14:55:39 +01:00
Emmanuel Marty	53fcd3b1a8	Generalize merging very large matches	2019-10-24 13:05:32 +02:00
Emmanuel Marty	05d77095ca	Bump version	2019-10-22 12:39:27 +02:00
Emmanuel Marty	b84fe7c332	Further increase LZSA2 ratio by ~0.1% on average	2019-10-22 12:37:46 +02:00
Emmanuel Marty	2926ad8436	Remove unused #includes	2019-10-21 12:29:38 +02:00
Emmanuel Marty	d9156d3d2b	Reduce LZSA1 token count by 2.5% on average	2019-10-19 13:10:41 +02:00
Emmanuel Marty	89f1664ae6	Remove unused code	2019-10-11 09:14:19 +02:00
Emmanuel Marty	c363ecf527	Remove unused code	2019-10-11 09:11:49 +02:00
Emmanuel Marty	5141ed7c59	Remove unused code	2019-10-11 09:11:41 +02:00
Emmanuel Marty	4436f216ce	Bump version	2019-10-11 09:06:50 +02:00
Emmanuel Marty	baa53f6889	Newly compressed LZSA2 files depack 0.7% faster	2019-10-11 09:05:58 +02:00
Emmanuel Marty	b5117c3dfe	Fixes for -stats	2019-10-11 00:25:46 +02:00
Emmanuel Marty	d209b73a30	Fix small bug	2019-10-10 14:42:08 +02:00
Emmanuel Marty	c1b18fb9fd	Implement -stats	2019-10-09 18:20:22 +02:00
Emmanuel Marty	6ce846ff24	Speed up LZSA2 compression	2019-10-09 16:07:29 +02:00
Emmanuel Marty	b09dadb1c1	Small LZSA2 token count reduction	2019-10-09 13:16:29 +02:00
Emmanuel Marty	03f841d04f	Speed up LZSA2 compression	2019-10-08 20:26:21 +02:00
Emmanuel Marty	44df8f3d2d	Add early-out, speed LZSA2 compression up further	2019-10-08 16:23:33 +02:00
Emmanuel Marty	bfb383befd	Speed up LZSA2 compression	2019-10-08 09:39:18 +02:00
Emmanuel Marty	39e2a90f81	Prevent small matchfinder inefficiency	2019-10-04 11:54:54 +02:00
Emmanuel Marty	33327201f7	Fix small LZSA2 token reduction inefficiency	2019-10-03 16:58:34 +02:00
Emmanuel Marty	29c6f3b2a3	Remove erroneous else statement	2019-09-26 19:13:09 +02:00
Emmanuel Marty	e55c80a475	Clean up use of MODESWITCH_PENALTY; bump version	2019-09-24 14:43:17 +02:00
Emmanuel Marty	de0ff5d3b0	Reduce memory used for compression	2019-09-24 00:21:17 +02:00
Emmanuel Marty	249b8a4c46	Increase LZSA2 ratio and use forward parser for -m	2019-09-23 20:24:50 +02:00
Emmanuel Marty	74040890fc	Speed up LZSA2 compression (same binary output)	2019-09-23 16:58:03 +02:00
Emmanuel Marty	81e15d10f0	Add extra safety checks to LZSA2 token reducer	2019-09-22 20:41:09 +02:00
Emmanuel Marty	1869d85c1f	Simplify LZSA1 token reducer (same binary output)	2019-09-22 20:34:08 +02:00
Emmanuel Marty	1a4f662360	Bump version	2019-09-20 12:26:16 +02:00
Emmanuel Marty	c12e20b7fb	Improve LZSA2 compression ratio	2019-09-20 12:24:27 +02:00
Emmanuel Marty	51644ad2f9	Speed LZSA2 compression up further; fix typo	2019-09-19 17:18:37 +02:00
Emmanuel Marty	1495b27f69	Speed up LZSA1 compression with forward arrivals	2019-09-19 12:57:39 +02:00
Emmanuel Marty	c052a188f2	Reduce LZSA2 forward arrivals memory use	2019-09-19 11:46:03 +02:00
Emmanuel Marty	e4076e4090	Speed LZSA2 compression up; tiny ratio increase	2019-09-19 00:11:26 +02:00
Emmanuel Marty	8b7d0ab04d	Increase LZSA2 ratio. Decrease token count	2019-09-17 08:10:52 +02:00
Emmanuel Marty	b1da9c1aee	Add extra bound checks in C decompressors	2019-09-12 16:19:14 +02:00
Francois Berder	4f2d7da136	Fix main return value if compressing ... Signed-off-by: Francois Berder <18538310+francois-berder@users.noreply.github.com>	2019-08-28 09:41:54 +01:00
Francois Berder	a318ac2f83	Fix memory leak in comparestream_open ... Signed-off-by: Francois Berder <18538310+francois-berder@users.noreply.github.com>	2019-08-28 09:40:49 +01:00
Francois Berder	da67938978	Set dictionnary to NULL in lzsa_dictionary_free ... Signed-off-by: Francois Berder <18538310+francois-berder@users.noreply.github.com>	2019-08-28 09:39:07 +01:00
Emmanuel Marty	2d213bcff1	Bump version number	2019-08-27 13:18:23 +02:00
Emmanuel Marty	ef259e6867	Implement forward arrivals optimal parsers	2019-08-27 00:51:34 +02:00
Emmanuel Marty	1b368e71ad	Fix comments, header single inclusion defines	2019-08-04 16:42:30 +02:00
Emmanuel Marty	b3aae36ecc	Bump version	2019-07-28 00:25:51 +02:00
Emmanuel Marty	b613d01565	Test incompressible data with raw blocks	2019-07-26 13:30:41 +02:00
Emmanuel Marty	316dfdcdce	Fix comments, remove unused vars	2019-07-26 01:12:17 +02:00
Emmanuel Marty	4835e4c26c	Support backward decompression	2019-07-24 20:08:23 +02:00
Emmanuel Marty	0c4dbf2b72	Add files via upload ... Show decompression safety distance for raw blocks	2019-07-24 15:43:44 +02:00
Emmanuel Marty	9f313d6ee6	Handle EOD in C depacker; fix #18; fix typos in usage	2019-07-23 23:28:52 +02:00
Emmanuel Marty	2d65f2a7c8	Bump version	2019-07-14 10:15:10 +02:00
Emmanuel Marty	12086b974a	Fix comments, remove unused vars	2019-07-08 17:57:36 +02:00
Emmanuel Marty	59eb891394	Improve LZSA2 compression ratio further and allow incompressible raw blocks	2019-07-01 09:25:19 +02:00
Emmanuel Marty	3c8c4283c7	Use real cost of rep offset when considering a reduction; fix small bug in repmatch handling when doing reductions.	2019-06-26 13:55:00 +02:00
Emmanuel Marty	2a265f9cf9	Remove erroneous comment	2019-06-25 11:13:13 +02:00
Emmanuel Marty	7867618f87	Fix matchfinder limitation	2019-06-25 11:09:19 +02:00
Emmanuel Marty	63c261ce2d	Small improvement of the LZSA2 compression ratio	2019-06-21 17:28:17 +02:00
Emmanuel Marty	71f262081b	Use -m3 as an upper bound for LZSA2	2019-06-11 10:38:47 +02:00
Emmanuel Marty	43ff88477f	Update tool version number	2019-06-10 14:09:23 +02:00
Emmanuel Marty	d564c1ecb8	Fixes	2019-06-08 19:03:33 +02:00
Emmanuel Marty	2e48e926a1	Fix self-tests, in-memory benchmarks in raw mode	2019-06-08 18:05:00 +02:00
Emmanuel Marty	659f6c14a9	Small improvement over 6502 LZSA2 depacker	2019-06-08 16:18:36 +02:00
Emmanuel Marty	6e1201e221	Moved	2019-06-08 13:36:40 +02:00
Emmanuel Marty	3008d47012	Moved	2019-06-08 13:36:20 +02:00
Emmanuel Marty	79ed7bf91e	Further update LZSA2 format; avoid name conflicts	2019-06-08 13:35:03 +02:00
Emmanuel Marty	ddd2ac4c56	Remove file after restructuring	2019-06-07 23:18:52 +02:00
Emmanuel Marty	8ef0396b50	Remove file after restructuring	2019-06-07 23:18:43 +02:00
Emmanuel Marty	ee6ab423fc	Remove file after restructuring	2019-06-07 23:18:14 +02:00
Emmanuel Marty	59d2383b03	Remove file after restructuring	2019-06-07 23:18:05 +02:00
Emmanuel Marty	74e8d82231	Remove file after restructuring	2019-06-07 23:17:56 +02:00
Emmanuel Marty	026537f7f2	Remove file after restructuring	2019-06-07 23:17:47 +02:00
Emmanuel Marty	998714d714	Remove file after restructuring	2019-06-07 23:17:37 +02:00
Emmanuel Marty	124adf8ade	Remove file after restructuring	2019-06-07 23:17:16 +02:00
Emmanuel Marty	59d7074da1	Remove file after restructuring	2019-06-07 23:17:07 +02:00
Emmanuel Marty	e924390c8e	Remove file after restructuring	2019-06-07 23:16:47 +02:00
Emmanuel Marty	47315447fd	Remove file after restructuring	2019-06-07 23:16:29 +02:00
Emmanuel Marty	b4e3c07d3a	Split code, add automated tests, update LZSA2	2019-06-07 23:15:40 +02:00
emmanuel-marty	45cb124c4d	Simplify and speed up in-tool decompressors; add in-memory benchmark	2019-05-17 08:57:01 +02:00
emmanuel-marty	a708a02048	Fix compilation warning	2019-05-14 12:18:35 +02:00
emmanuel-marty	277b5b1025	Move top-level streaming compression code to library	2019-05-13 22:22:53 +02:00
emmanuel-marty	0f4ffa7166	Clean up check for emitting a single raw block	2019-05-12 23:21:50 +02:00
emmanuel-marty	c96594680a	Use full BLOCK_SIZE for dictionary, and use last BLOCK_SIZE bytes rather than first	2019-05-12 14:44:28 +02:00
emmanuel-marty	3caa72c722	Clarify optimizer	2019-05-11 11:42:18 +02:00
emmanuel-marty	8b7b4a2b4f	Check in LZSA2 implementation (ratio competitive with ZX7, faster decompression)	2019-05-09 16:51:29 +02:00
emmanuel-marty	49b0739050	Isolate frame implementation details	2019-05-03 19:17:14 +02:00
emmanuel-marty	31ededcdf7	Rejoin large patterns	2019-05-03 13:45:37 +02:00
emmanuel-marty	bab5225e4c	Implement dictionary support	2019-05-02 18:38:57 +02:00
emmanuel-marty	ef6b43a296	When using -m, decompress ~7% faster by trading ~0.5% of compression ratio	2019-05-02 11:23:57 +02:00
emmanuel-marty	9e75a55dd3	Reduce token count by 1% without changing the ratio	2019-04-25 13:01:56 +02:00
emmanuel-marty	2b9780bd65	Finalize lzsa1 compressed format, speed up and simplify decompression	2019-04-24 09:47:40 +02:00
emmanuel-marty	593110ae5d	Fix parsing of -m option	2019-04-22 14:16:25 +02:00
emmanuel-marty	331d6f9911	Add --prefer-ratio, --prefer-speed, -m options	2019-04-21 09:41:12 +02:00
emmanuel-marty	6c4a86c9fe	Make compression 5% faster	2019-04-20 16:24:54 +02:00
emmanuel-marty	fb79d319cb	Speed up portable decompressor a bit	2019-04-20 10:27:24 +02:00
emmanuel-marty	3e5639afaf	Reduce token count further	2019-04-20 10:26:45 +02:00
emmanuel-marty	88054841cf	Add strict block checks when decompressing, comparing	2019-04-20 10:26:13 +02:00
emmanuel-marty	aa5dbe283d	Small optimization to the cost model	2019-04-13 12:33:39 +02:00
emmanuel-marty	4913fe5d48	Fix reporting of compressed size for raw mode	2019-04-13 12:33:25 +02:00
emmanuel-marty	cf51f1775a	Fix small literals cost estimate issue in optimizer, gain a small amount of compression and reduction of token count	2019-04-11 23:47:57 +02:00
emmanuel-marty	2451d5802e	Fix typo in lzsa tool	2019-04-10 17:38:22 +02:00
emmanuel-marty	9649073131	Speed up compression	2019-04-08 09:44:07 +02:00
emmanuel-marty	837b6895cf	Further reduce the number of tokens for even better decompression speed; increase compression slightly	2019-04-07 15:10:53 +02:00
emmanuel-marty	9c35d5115b	When verbose, display uncompressed bytes/token ratio	2019-04-07 15:10:17 +02:00
emmanuel-marty	6aa2dae4b3	Add context to libdivsufsort, don't allocate memory during compression	2019-04-07 00:01:22 +02:00
emmanuel-marty	1353573af1	Small cleanup for end-of-data handling in decompression, check commands	2019-04-06 00:02:11 +02:00
emmanuel-marty	a785010448	Revert token to O|LLL|MMMM; revert to always shifting the match offset by 1; set raw block end marker as a large zero-size match	2019-04-05 23:16:05 +02:00
emmanuel-marty	06e6a14871	Add optimization pass to reduce the number of command tokens in the compressed data blocks without changing the compression ratio	2019-04-05 16:32:11 +02:00
emmanuel-marty	f05359b63d	Don't write an unnecessary footer byte when emitting a raw block	2019-04-05 12:13:51 +02:00
emmanuel-marty	1ef1ad8111	Reorganize token byte for faster decoding on 8-bit CPUs, without affecting the compression ratio	2019-04-05 11:58:44 +02:00
emmanuel-marty	c7692cf688	Store 16-bit lengths and match offsets directly, to simplify decompression on 8-bit CPUs without affecting the compression ratio	2019-04-05 10:42:06 +02:00
emmanuel-marty	c86d38ba63	Reduce the number of literals required at the end of a compressed block	2019-04-05 09:28:16 +02:00
emmanuel-marty	18fc4da994	Implement raw block mode	2019-04-03 13:05:10 +02:00
emmanuel-marty	11d1ff8cd7	Use 3-byte file header	2019-04-03 11:26:36 +02:00
emmanuel-marty	1f04705845	Fix degenerate case; use full 32 bits for suffix array intervals; make EOD parsable by a decompressor as a long 0 match offset as well; use more aggressive compression settings.	2019-04-03 10:16:12 +02:00
emmanuel-marty	fcfdbe9745	Add autodocs to internal compressor functions	2019-04-02 15:03:21 +02:00
emmanuel-marty	8b992bb33a	Add autodocs to public functions in compressor and decompressor	2019-04-02 12:12:12 +02:00
marty-emmanuel	e216b0c544	Initial checkin	2019-04-01 18:04:56 +02:00