Commit Graph

119 Commits

Author SHA1 Message Date
dgelessus
5c3bc5d7e5 Remove custom stream types and read all resource data upfront again
The custom stream types were almost always slower than just reading the
entire data into memory, and there's no reason not to do that -
resources are small enough that memory usage and disk IO speed aren't a
concern (at least not for any machine that's modern enough to run
Python 3...).

Perhaps the only performance advantage was when reading a small amount
of data from the start of a compressed resource. In that case the
custom stream could incrementally decompress only the part of the data
that's actually needed, which was a bit faster than decompressing the
entire resource and then throwing away most of the data. But this
situation is rare enough that it's not worth handling in the rsrcfork
library. If this is a real performance issue for someone, they can
manually call the incremental decompression functions from
rsrcfork.compress where needed.
2020-11-01 19:28:25 +01:00
dgelessus
d74dbc41ba Remove no longer needed type: ignore comment from .__main__ 2020-11-01 19:13:11 +01:00
dgelessus
9e6dfacff6 Add custom stream type for compressed resources 2020-08-01 14:11:06 +02:00
dgelessus
8d39469e6e Wrap _io_utils.SubStream in an io.BufferedReader for performance
Although the underlying stream is already buffered, the extra
BufferedReader wrapper around the SubStream results in a noticeable
performance improvement.
2020-07-23 15:49:28 +02:00
dgelessus
028be98e8d Suppress incorrect mypy shutil.copyfileobj error (python/mypy#8962) 2020-07-23 13:31:57 +02:00
dgelessus
98551263b3 Work around false mypy error about SubStream.__enter__ 2020-07-23 13:21:34 +02:00
dgelessus
0054d0f7b5 Fix variable naming conflict in show_filtered_resources
mypy does not like it when the same variable name is used for different
types, even if it is in unrelated if branches and could never actually
cause any problems.
2020-07-23 13:17:02 +02:00
dgelessus
126795239c Reimplement Resource.data_raw using a custom stream type (SubStream)
This way all reads performed on a resource data stream are forwarded
to the underlying resource file stream, with the read offsets and
lengths adjusted appropriately.
2020-07-23 02:42:32 +02:00
dgelessus
2907d9f9e8 Rewrite __main__ code to use stream-based resource reading 2020-07-21 14:45:48 +02:00
dgelessus
5c96baea29 Change (raw_)hexdump to yield lines instead of printing directly 2020-07-21 14:27:43 +02:00
dgelessus
664e992fa3 Rewrite Resource methods using stream API where appropriate 2020-07-21 14:20:50 +02:00
dgelessus
61247ec783 Add initial API and tests for stream-based resource reading
For now the stream-based API is a simple BytesIO wrapper around
data/data_raw, but it will be optimized in the future.
2020-07-21 14:12:09 +02:00
dgelessus
0f6018e4bf Move .compress.common.make_peekable and related code into ._io_utils 2020-07-19 23:16:36 +02:00
dgelessus
476a68916b Merge implementations of read_exact functions/methods
The old functions/methods still exist, so that they continue to raise
the same exceptions as before (which are different depending on
context), but they now use the same implementation internally.
2020-07-18 21:07:12 +02:00
dgelessus
4bbf2f7c14 Bump version to 1.8.1.dev 2020-07-18 17:47:48 +02:00
dgelessus
82b5926b4f Release version 1.8.0 2020-07-18 17:31:25 +02:00
dgelessus
d367a9238a Add bytes_quote helper function
bytes_quote does the same as bytes_escape, but automatically adds the
quote character around the escaped string.
2020-07-07 01:03:38 +02:00
dgelessus
33c4016124 Fix flake8 problems 2020-07-07 00:04:54 +02:00
dgelessus
b01cfc77cf Don't pass required=True to add_subparsers
The required kwarg of add_subparsers was only added in Python 3.7, and
we currently still support Python 3.6.
2020-07-07 00:01:57 +02:00
dgelessus
a9f54b678c Add py.typed marker file for PEP 561
This allows type checkers like mypy to use the type hints in our code
when type-checking another project that imports this library.
2020-07-06 23:57:25 +02:00
dgelessus
b46018e666 Use is_printable in definition of _TRANSLATE_NONPRINTABLES 2020-07-06 18:01:34 +02:00
dgelessus
b0eefe3889 Replace custom CLI subcommand system with standard argparse subparsers
This is a purely internal change and should have no visible effect on
the command-line interface.
2020-07-05 19:43:01 +02:00
dgelessus
ba284d1800 Fix a bunch of flake8 violations 2019-12-30 03:00:12 +01:00
dgelessus
6adf8eb88d Fix mypy errors about byte strings as format string parameters 2019-12-30 01:48:33 +01:00
dgelessus
e132a91dea Fix missing sys.exit calls in CLI subcommand functions 2019-12-30 01:48:33 +01:00
dgelessus
4e1cd05412 Fix miscellaneous mypy errors 2019-12-30 01:48:33 +01:00
dgelessus
8fc24040ea Add resource-info subcommand 2019-12-26 01:58:23 +01:00
dgelessus
d492d9a6a8 Remove an incorrect assertion from describe_resource
red.compressed_info can be None here if decompress is False.
2019-12-26 01:50:34 +01:00
dgelessus
d0e1eaf262 Add raw-compress-info subcommand (#6) 2019-12-26 00:34:27 +01:00
dgelessus
1e55569442 Add support for passing filters to the list subcommand 2019-12-25 01:47:03 +01:00
dgelessus
2abf6e2a06 Add class for resource filters in place of lambdas
This is easier to debug (printing out a lambda doesn't show what values
it checks against) and makes it easier to check that the filter values
are valid.
2019-12-25 00:15:35 +01:00
dgelessus
2b0bbb19ed Refactor filter_resources in __main__
With the new implementation, each filter is converted to a function,
then all resources are checked if they match any of the filter
functions. This is simpler than the old implementation, where the
resource lookup code was slightly different for some filter forms.
2019-12-25 00:15:35 +01:00
dgelessus
c009e8f80f Support passing an empty filter list to filter_resources 2019-12-25 00:15:35 +01:00
dgelessus
d67641d537 Remove compatibility code for old CLI syntax 2019-12-25 00:15:30 +01:00
dgelessus
b2502c48a2 Bump version to 1.7.1.dev 2019-12-17 12:16:39 +01:00
dgelessus
158ca4884b Release version 1.7.0 2019-12-17 11:28:26 +01:00
dgelessus
8568f355c4 Remove incorrect outdated paragraph from list subcommand help 2019-12-10 16:15:18 +01:00
dgelessus
97d2dbe1b3 Change formatting of command help strings in source code
The automatic textwrap.dedent makes it impossible to cleanly extract
parts of the help strings into separate constants.
2019-12-10 15:58:20 +01:00
dgelessus
a4b6328782 Fix 'dcmp' (0) jump table decompression for large segment numbers 2019-12-04 23:36:57 +01:00
dgelessus
393160b5da Add raw-decompress subcommand (#6) 2019-12-04 23:36:56 +01:00
dgelessus
476eaecd17 Fix typo in the help text for rsrcfork read 2019-12-04 21:16:29 +01:00
dgelessus
546edbc31a Update and improve resource and resource map reprs 2019-12-04 02:01:40 +01:00
dgelessus
cf6ce3c2a6 Move _LazyResourceMap out of ResourceFile 2019-12-04 02:01:40 +01:00
dgelessus
af2ac70676 Simplify ResourceFile._references and ._LazyResourceMap
The _references map now stores Resource objects directly, instead of
constructing them only when they are looked up. Resource objects are
now lazy themselves, so the previous lazy resource creation mechanism
is redundant.

_LazyResourceMap is now a simple read-only wrapper around an existing
map. The custom class is now only used to provide a specialized repr.
2019-12-04 02:01:40 +01:00
dgelessus
5af455992b Refactor resource reading internals
The reading of resource name and data is now performed in the Resource
class (lazily, when the respective attributes are accessed) instead of
in ResourceFile._LazyResourceMap.
2019-12-04 02:01:40 +01:00
dgelessus
2193c81518 Bump version to 1.6.1.dev 2019-12-04 01:45:15 +01:00
dgelessus
7dc0d980a3 Release version 1.6.0 2019-12-04 01:35:57 +01:00
dgelessus
ec5eb3bcc1 Don't display header data and attributes in list output
This is redundant now that there is a dedicated info subcommand.
2019-10-22 13:26:03 +02:00
dgelessus
25bec2f93a Add info subcommand to display technical info/stats about resource file 2019-10-22 13:18:25 +02:00
dgelessus
6fbb919285 Display warnings when the old CLI syntax is used 2019-10-22 10:46:25 +02:00