Commit Graph

70 Commits

Author SHA1 Message Date
Denys Vlasenko
cfcd2399b2 make xmalloc_open_zipped_read_close result NUL terminated
Compat with xmalloc_open_read_close

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2014-12-07 00:49:55 +01:00
Denys Vlasenko
b4c11c1397 libarchive: add capability to unpack to mem.buffer
The performance and number of processes for a "depmod -a" with gzipped
modules was abysmal. This patch adds a code path without fork,
benefiting all users of xmalloc_open_zipped_read_close.

"modinfo radeon.ko.gz", a single-file reader, got 30% faster.
"depmod -a", which used to fork over 800 times, got 20% faster.

Heavily based on a patch by Lauri Kasanen <curaga@operamail.com>

function                                             old     new   delta
setup_transformer_on_fd                                -     159    +159
transformer_write                                      -     122    +122
fork_transformer                                       -     112    +112
xmalloc_open_zipped_read_close                        63     118     +55
read_bunzip                                         1866    1896     +30
xtransformer_write                                     -      19     +19
unzip_main                                          2449    2462     +13
bbunpack                                             755     766     +11
unpack_lzma_stream                                  2717    2723      +6
unpack_xz_stream                                    2393    2397      +4
unpack_Z_stream                                     1173    1175      +2
inflate_unzip                                        111     105      -6
check_signature16                                     70      63      -7
unpack_bz2_stream                                    359     349     -10
unpack_unxz                                           12       -     -12
unpack_unlzma                                         12       -     -12
unpack_uncompress                                     12       -     -12
unpack_gunzip                                         12       -     -12
unpack_bunzip2                                        12       -     -12
open_transformer                                     106      92     -14
inflate_unzip_internal                              1945    1916     -29
unpack_gz_stream                                     693     655     -38
open_zipped                                           89      47     -42
setup_unzip_on_fd                                    142      53     -89
------------------------------------------------------------------------------
(add/remove: 4/5 grow/shrink: 7/8 up/down: 533/-295)          Total: 238 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2014-12-07 00:44:00 +01:00
Denys Vlasenko
e7800f351a Rename transformer_aux_data_t -> transformer_state_t
No code changes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2014-12-07 00:42:49 +01:00
Denys Vlasenko
a9dc7c2f59 lzop: add overflow check
See CVE-2014-4607
http://www.openwall.com/lists/oss-security/2014/06/26/20

function                                             old     new   delta
lzo1x_decompress_safe                               1010    1031     +21

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2014-06-30 10:14:34 +02:00
Denys Vlasenko
81071e6872 unlzma: add comments about possible bug from BZ 2689
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2014-02-28 15:42:10 +01:00
Denys Vlasenko
2108a6f0b5 unlzma: move some variables in "more local" scope
No code changes as verified by objdump

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2014-02-28 15:05:43 +01:00
Denys Vlasenko
640ce3de07 zcat: complain if input is not compressed
function                                             old     new   delta
buffer_fill_and_print                                178     191     +13
varvalue                                             735     743      +8
bbunpack                                             747     755      +8
open_zipped                                           85      89      +4
xmalloc_open_zipped_read_close                        61      63      +2
get_addr_1                                           240     242      +2
fbsplash_main                                       1228    1230      +2
pstree_main                                          322     321      -1
builtin_type                                         121     119      -2
do_load                                              954     926     -28
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 7/3 up/down: 39/-31)              Total: 8 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2014-02-02 02:06:38 +01:00
Denys Vlasenko
b664f740d9 libbb: open_zipped() should not fail on non-compressed files
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2014-01-27 13:02:18 +01:00
Denys Vlasenko
0f592d7fb9 tar: tighten up pax header validity check
function                                             old     new   delta
get_header_tar                                      1785    1795     +10

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2014-01-10 18:02:38 +01:00
Denys Vlasenko
7c47b560a8 libarchive: open_zipped() does not need to check extensions for e.g. gzip
We only need to check for signature-less extensions,
currently only .lzma. The rest can be happily autodetected.

This fixes "zcat FILE_WITHOUT_GZ_EXT" case, among others.

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2014-01-10 14:12:11 +01:00
Denys Vlasenko
9b2a9f0210 fix assorted unused code and wrong format specs found by cppchekc (bug 6716)
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-11-29 16:43:33 +01:00
Denys Vlasenko
ebfa9b5aa1 tar: fix exitcode check for MMU-spawned unpacking helpers
Testcase: tar xvzf EMPTY_FILE

function                                             old     new   delta
open_transformer                                     102     106      +4
get_header_tar                                      1781    1785      +4

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-11-19 14:44:04 +01:00
Denys Vlasenko
5117eff6f9 Fix some compiler warnings emitted by gcc-4.8.0
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-10-16 14:21:20 +02:00
Denys Vlasenko
932e233a49 bunzip2: fix off-by-one check
stage3-armv7a_hardfp-20130209.tar.bz2, 149189948 bytes long,
md5sum b29ce23312e14eb15a143377d4a38473, was failing to unpack.

It so happened that this file has a run which exactly fills
the 90k buffer. The check was "size >= bufsize", apparently
it has to be ">".

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-10-06 22:53:14 +02:00
Denys Vlasenko
a613aa1b4c ar: better comment
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-09-10 16:27:08 +02:00
Peter Korsgaard
2a053a2430 ar: read_num(): fix reading fields using the entire width
ar fields are fixed length text strings (padded with spaces). Ensure
bb_strtou doesn't read past the field in case the full width is used.

The fields are only read once, so the simplest/smallest solution to me
seems to be to just pass the length to read_num() and then zero terminate
the string before passing it to bb_strtou. This does mean that the fields
MUST be read in reverse order, so some minor reshuffling was needed.

Bloat-o-meter:
function                                             old     new   delta
get_header_ar                                        394     414     +20
read_num                                              29      36      +7
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 2/0 up/down: 27/0)               Total: 27 bytes

Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-09-10 16:22:12 +02:00
Rich Felker
16614e9bab *: change execl sentinels from NULL to (char*)0
Signed-off-by: Rich Felker <dalias@aerifal.cx>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-06-30 13:45:17 +02:00
Denys Vlasenko
e0a6ab698f Fix build failures caused by not compiling open_transformer.c (#2)
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-05-15 03:23:30 +02:00
Denys Vlasenko
abbc478f1c Fix build failures caused by not compiling open_transformer.c
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-05-12 02:34:38 +02:00
Denys Vlasenko
507f6ea6d2 decompress_unlzma: move function, no code changes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-03-01 14:48:10 +01:00
Denys Vlasenko
a2d04e0702 decompress_unlzma: 10% speedup in "small" code
text	   data	    bss	    dec	    hex	filename
   1796	      0	      0	   1796	    704	decompress_unlzma.o
   1801	      0	      0	   1801	    709	decompress_unlzma.o

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-03-01 14:43:07 +01:00
Denys Vlasenko
c09fd27c0a decompress_unlzma: make "fast" version a bit smaller
It is not slower. In fact it seems a tiny bit faster too.

   text	   data	    bss	    dec	    hex	filename
   2827	      0	      0	   2827	    b0b	decompress_unlzma.o
   2797	      0	      0	   2797	    aed	decompress_unlzma.o

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-03-01 14:37:58 +01:00
Denys Vlasenko
577235dee8 code shrink in check_errors_in_children()
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-28 16:38:25 +01:00
Leonid Lisovskiy
f59d563399 xz: fix put_unaligned_{l,b}e32
Signed-off-by: Leonid Lisovskiy <lly.dev@gmail.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 18:32:58 +01:00
Lasse Collin
380c8a0763 xz: support concatenated .xz streams
function                                             old     new   delta
xz_dec_reset                                           -      77     +77
unpack_xz_stream                                    2402    2397      -5

Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 17:26:40 +01:00
Lasse Collin
433757413f xz: mention xzminidec.c in README
Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 16:41:36 +01:00
Lasse Collin
c3045edec2 xz: fix incorrect XZ_BUF_ERROR
xz_dec_run() could incorrectly return XZ_BUF_ERROR if
all of the following was true:

  - The caller knows how many bytes of output to expect
    and only provides that much output space.

  - When the last output bytes are decoded, the
    caller-provided input buffer ends right before
    the LZMA2 end of payload marker. So LZMA2 won't
    provide more output anymore, but it won't know it
    yet and thus won't return XZ_STREAM_END yet.

  - A BCJ filter is in use and it hasn't left any
    unfiltered bytes in the temp buffer. This can happen
    with any BCJ filter, but in practice it's more likely
    with filters other than the x86 BCJ.

This fixes <https://bugzilla.redhat.com/show_bug.cgi?id=735408>
where Squashfs thinks that a valid file system is corrupt.
Thanks to Jindrich Novy for telling me that such a bug report
exists, Phillip Lougher for providing excellent debug info,
and other people on #fedora-ppc.

This also fixes a similar bug in single-call mode where the
uncompressed size of a XZ Block using BCJ + LZMA2 was 0 bytes
and caller provided no output space. Many empty .xz files
don't contain any Blocks and thus don't trigger this bug.

This also tweaks a closely related detail: xz_dec_bcj_run()
could call xz_dec_lzma2_run() to decode into temp buffer when
it was known to be useless. This was harmless although it
wasted a minuscule number of CPU cycles.

Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 16:39:56 +01:00
Lasse Collin
a1ae2b75a7 xz: fix decoding of LZMA2 streams having no uncompressed data.
No .xz encoder creates files with empty LZMA2 streams,
but such files would still be valid and decompressors
must accept them.

Note that empty .xz files are a different thing than
empty LZMA2 streams. This bug didn't affect typical .xz
files that had no uncompressed data.

Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 16:38:06 +01:00
Lasse Collin
efb800439f cz: add C++ support to xz.h
Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 16:37:18 +01:00
Lasse Collin
aada3126a7 xz: remove an empty line from xz_dec_lzma2.c
Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 16:36:17 +01:00
Lasse Collin
b967e42b77 xz: make bcj_x86_test_msbyte() an inline function
Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 16:34:06 +01:00
Lasse Collin
18714d8460 xz: add a comment about using uint32_t as vli_type
Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 16:32:03 +01:00
Lasse Collin
9056fcecbf xz: avoid "NOTE:" in xz.h
Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 16:28:33 +01:00
Lasse Collin
3a7b2417f9 xz: update README
Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 16:26:03 +01:00
Lasse Collin
04f296b28a xz: omit explicit \0 from HEADER_MAGIC
Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-27 16:23:24 +01:00
Denys Vlasenko
2aec773688 rpm: use "create+rename" method of replacing existing files
Users were reporting getting errors like
"ls: error while loading shared libraries: libc.so.6: ELF load command past end of file"
while rpm was unpacking glibc tarball.

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-02-20 15:58:42 +01:00
Denys Vlasenko
60cb48ca50 whitespace cleanup. no code changes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2013-01-14 15:57:44 +01:00
Denys Vlasenko
4d5955e9ec decompress_uncompress: comment out a bigger chunk of debug code
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-12-05 11:08:30 +01:00
Denys Vlasenko
a8461173ba decompress_uncompress: comment out debug printout on corrupted data
99% plus of all people who'll get corrupted archive wouldn't bother
debugging it. The rest can uncomment the code.

function                                             old     new   delta
unpack_Z_stream                                     1304    1234     -70

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-12-05 01:06:05 +01:00
Denys Vlasenko
c71547ccfc decompress_uncompress: move 'code' variable into loop - sole user
Apparently, gcc does this optimization itself, since generated code is the same.

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-12-05 01:04:54 +01:00
Denys Vlasenko
440a509849 dpkg: fix creation of .list files (were empty since b768aeb). Closes 5324
While at it, fix filename order and free the list of names.

function                                             old     new   delta
llist_rev                                              -      21     +21
get_header_tar                                      1733    1741      +8
unpack_package                                       587     585      -2
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 1/1 up/down: 29/-2)              Total: 27 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-06-22 16:27:21 +02:00
Denys Vlasenko
d52c9510fd trivial small speed optimization
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-06-22 15:54:05 +02:00
Etienne Le Sueur
cfc212cdff tar: fix 256-bit encoded number decoding
Signed-off-by: Etienne Le Sueur <elesueur@vmware.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-06-09 08:37:05 +02:00
Anthony G. Basile
6e9284db78 build system: fix build failure when only gunzip is selected
Signed-off-by: Anthony G. Basile <blueness@gentoo.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-05-20 12:56:17 +02:00
Denys Vlasenko
d133144d41 fix build failure when compressed help is selected, but bz2 compression is not
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-04-29 14:28:12 +02:00
Denys Vlasenko
df9196d824 Fix another build failure found with randconfig
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-04-20 18:59:14 +02:00
Denys Vlasenko
8cab66730a fix build breakage found by randconfig
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-04-20 14:48:00 +02:00
Denys Vlasenko
d29c946f93 fix trivial bug in unpack_gz_stream (wrong fd used in read)
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-03-06 16:46:25 +01:00
Denys Vlasenko
b5d6ae9a33 Don't compile get_header_tar_FOO function if they are not needed
Now get_header_tar_gz, get_header_tar_bz2, get_header_tar_lzma
are only used if dpkg is built.

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-03-06 16:34:23 +01:00
Denys Vlasenko
faac1d3e6e tar,rpm2cpio: check that child decompressor did not error out
function                                             old     new   delta
check_errors_in_children                               -      57     +57
tar_main                                             833     848     +15
get_header_tar                                      1720    1733     +13
rpm2cpio_main                                        147     140      -7
handle_SIGCHLD                                        41       -     -41
------------------------------------------------------------------------------
(add/remove: 1/1 grow/shrink: 2/1 up/down: 85/-48)             Total: 37 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2012-03-06 16:33:42 +01:00