fixed #14: add check for nonstandard dos33 sector size and reset if found

* used when restoring segment parser from json

LICENSE

@@ -1,339 +1,373 @@
-		    GNU GENERAL PUBLIC LICENSE
+Mozilla Public License Version 2.0
-		       Version 2, June 1991
+==================================
 
- Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
+1. Definitions
- 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+--------------
- Everyone is permitted to copy and distribute verbatim copies
- of this license document, but changing it is not allowed.
 
-			    Preamble
+1.1. "Contributor"
+    means each individual or legal entity that creates, contributes to
+    the creation of, or owns Covered Software.
 
-  The licenses for most software are designed to take away your
+1.2. "Contributor Version"
-freedom to share and change it.  By contrast, the GNU General Public
+    means the combination of the Contributions of others (if any) used
-License is intended to guarantee your freedom to share and change free
+    by a Contributor and that particular Contributor's Contribution.
-software--to make sure the software is free for all its users.  This
-General Public License applies to most of the Free Software
-Foundation's software and to any other program whose authors commit to
-using it.  (Some other Free Software Foundation software is covered by
-the GNU Lesser General Public License instead.)  You can apply it to
-your programs, too.
 
-  When we speak of free software, we are referring to freedom, not
+1.3. "Contribution"
-price.  Our General Public Licenses are designed to make sure that you
+    means Covered Software of a particular Contributor.
-have the freedom to distribute copies of free software (and charge for
-this service if you wish), that you receive source code or can get it
-if you want it, that you can change the software or use pieces of it
-in new free programs; and that you know you can do these things.
 
-  To protect your rights, we need to make restrictions that forbid
+1.4. "Covered Software"
-anyone to deny you these rights or to ask you to surrender the rights.
+    means Source Code Form to which the initial Contributor has attached
-These restrictions translate to certain responsibilities for you if you
+    the notice in Exhibit A, the Executable Form of such Source Code
-distribute copies of the software, or if you modify it.
+    Form, and Modifications of such Source Code Form, in each case
+    including portions thereof.
 
-  For example, if you distribute copies of such a program, whether
+1.5. "Incompatible With Secondary Licenses"
-gratis or for a fee, you must give the recipients all the rights that
+    means
-you have.  You must make sure that they, too, receive or can get the
-source code.  And you must show them these terms so they know their
-rights.
 
-  We protect your rights with two steps: (1) copyright the software, and
+    (a) that the initial Contributor has attached the notice described
-(2) offer you this license which gives you legal permission to copy,
+        in Exhibit B to the Covered Software; or
-distribute and/or modify the software.
 
-  Also, for each author's protection and ours, we want to make certain
+    (b) that the Covered Software was made available under the terms of
-that everyone understands that there is no warranty for this free
+        version 1.1 or earlier of the License, but not also under the
-software.  If the software is modified by someone else and passed on, we
+        terms of a Secondary License.
-want its recipients to know that what they have is not the original, so
-that any problems introduced by others will not reflect on the original
-authors' reputations.
 
-  Finally, any free program is threatened constantly by software
+1.6. "Executable Form"
-patents.  We wish to avoid the danger that redistributors of a free
+    means any form of the work other than Source Code Form.
-program will individually obtain patent licenses, in effect making the
-program proprietary.  To prevent this, we have made it clear that any
-patent must be licensed for everyone's free use or not licensed at all.
 
-  The precise terms and conditions for copying, distribution and
+1.7. "Larger Work"
-modification follow.
+    means a work that combines Covered Software with other material, in 
+    a separate file or files, that is not Covered Software.
 
-		    GNU GENERAL PUBLIC LICENSE
+1.8. "License"
-   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+    means this document.
 
-  0. This License applies to any program or other work which contains
+1.9. "Licensable"
-a notice placed by the copyright holder saying it may be distributed
+    means having the right to grant, to the maximum extent possible,
-under the terms of this General Public License.  The "Program", below,
+    whether at the time of the initial grant or subsequently, any and
-refers to any such program or work, and a "work based on the Program"
+    all of the rights conveyed by this License.
-means either the Program or any derivative work under copyright law:
-that is to say, a work containing the Program or a portion of it,
-either verbatim or with modifications and/or translated into another
-language.  (Hereinafter, translation is included without limitation in
-the term "modification".)  Each licensee is addressed as "you".
 
-Activities other than copying, distribution and modification are not
+1.10. "Modifications"
-covered by this License; they are outside its scope.  The act of
+    means any of the following:
-running the Program is not restricted, and the output from the Program
-is covered only if its contents constitute a work based on the
-Program (independent of having been made by running the Program).
-Whether that is true depends on what the Program does.
 
-  1. You may copy and distribute verbatim copies of the Program's
+    (a) any file in Source Code Form that results from an addition to,
-source code as you receive it, in any medium, provided that you
+        deletion from, or modification of the contents of Covered
-conspicuously and appropriately publish on each copy an appropriate
+        Software; or
-copyright notice and disclaimer of warranty; keep intact all the
-notices that refer to this License and to the absence of any warranty;
-and give any other recipients of the Program a copy of this License
-along with the Program.
 
-You may charge a fee for the physical act of transferring a copy, and
+    (b) any new file in Source Code Form that contains any Covered
-you may at your option offer warranty protection in exchange for a fee.
+        Software.
 
-  2. You may modify your copy or copies of the Program or any portion
+1.11. "Patent Claims" of a Contributor
-of it, thus forming a work based on the Program, and copy and
+    means any patent claim(s), including without limitation, method,
-distribute such modifications or work under the terms of Section 1
+    process, and apparatus claims, in any patent Licensable by such
-above, provided that you also meet all of these conditions:
+    Contributor that would be infringed, but for the grant of the
+    License, by the making, using, selling, offering for sale, having
+    made, import, or transfer of either its Contributions or its
+    Contributor Version.
 
-    a) You must cause the modified files to carry prominent notices
+1.12. "Secondary License"
-    stating that you changed the files and the date of any change.
+    means either the GNU General Public License, Version 2.0, the GNU
+    Lesser General Public License, Version 2.1, the GNU Affero General
+    Public License, Version 3.0, or any later versions of those
+    licenses.
 
-    b) You must cause any work that you distribute or publish, that in
+1.13. "Source Code Form"
-    whole or in part contains or is derived from the Program or any
+    means the form of the work preferred for making modifications.
-    part thereof, to be licensed as a whole at no charge to all third
-    parties under the terms of this License.
 
-    c) If the modified program normally reads commands interactively
+1.14. "You" (or "Your")
-    when run, you must cause it, when started running for such
+    means an individual or a legal entity exercising rights under this
-    interactive use in the most ordinary way, to print or display an
+    License. For legal entities, "You" includes any entity that
-    announcement including an appropriate copyright notice and a
+    controls, is controlled by, or is under common control with You. For
-    notice that there is no warranty (or else, saying that you provide
+    purposes of this definition, "control" means (a) the power, direct
-    a warranty) and that users may redistribute the program under
+    or indirect, to cause the direction or management of such entity,
-    these conditions, and telling the user how to view a copy of this
+    whether by contract or otherwise, or (b) ownership of more than
-    License.  (Exception: if the Program itself is interactive but
+    fifty percent (50%) of the outstanding shares or beneficial
-    does not normally print such an announcement, your work based on
+    ownership of such entity.
-    the Program is not required to print an announcement.)
 
-These requirements apply to the modified work as a whole.  If
+2. License Grants and Conditions
-identifiable sections of that work are not derived from the Program,
+--------------------------------
-and can be reasonably considered independent and separate works in
-themselves, then this License, and its terms, do not apply to those
-sections when you distribute them as separate works.  But when you
-distribute the same sections as part of a whole which is a work based
-on the Program, the distribution of the whole must be on the terms of
-this License, whose permissions for other licensees extend to the
-entire whole, and thus to each and every part regardless of who wrote it.
 
-Thus, it is not the intent of this section to claim rights or contest
+2.1. Grants
-your rights to work written entirely by you; rather, the intent is to
-exercise the right to control the distribution of derivative or
-collective works based on the Program.
 
-In addition, mere aggregation of another work not based on the Program
+Each Contributor hereby grants You a world-wide, royalty-free,
-with the Program (or with a work based on the Program) on a volume of
+non-exclusive license:
-a storage or distribution medium does not bring the other work under
-the scope of this License.
 
-  3. You may copy and distribute the Program (or a work based on it,
+(a) under intellectual property rights (other than patent or trademark)
-under Section 2) in object code or executable form under the terms of
+    Licensable by such Contributor to use, reproduce, make available,
-Sections 1 and 2 above provided that you also do one of the following:
+    modify, display, perform, distribute, and otherwise exploit its
+    Contributions, either on an unmodified basis, with Modifications, or
+    as part of a Larger Work; and
 
-    a) Accompany it with the complete corresponding machine-readable
+(b) under Patent Claims of such Contributor to make, use, sell, offer
-    source code, which must be distributed under the terms of Sections
+    for sale, have made, import, and otherwise transfer either its
-    1 and 2 above on a medium customarily used for software interchange; or,
+    Contributions or its Contributor Version.
 
-    b) Accompany it with a written offer, valid for at least three
+2.2. Effective Date
-    years, to give any third party, for a charge no more than your
-    cost of physically performing source distribution, a complete
-    machine-readable copy of the corresponding source code, to be
-    distributed under the terms of Sections 1 and 2 above on a medium
-    customarily used for software interchange; or,
 
-    c) Accompany it with the information you received as to the offer
+The licenses granted in Section 2.1 with respect to any Contribution
-    to distribute corresponding source code.  (This alternative is
+become effective for each Contribution on the date the Contributor first
-    allowed only for noncommercial distribution and only if you
+distributes such Contribution.
-    received the program in object code or executable form with such
-    an offer, in accord with Subsection b above.)
 
-The source code for a work means the preferred form of the work for
+2.3. Limitations on Grant Scope
-making modifications to it.  For an executable work, complete source
-code means all the source code for all modules it contains, plus any
-associated interface definition files, plus the scripts used to
-control compilation and installation of the executable.  However, as a
-special exception, the source code distributed need not include
-anything that is normally distributed (in either source or binary
-form) with the major components (compiler, kernel, and so on) of the
-operating system on which the executable runs, unless that component
-itself accompanies the executable.
 
-If distribution of executable or object code is made by offering
+The licenses granted in this Section 2 are the only rights granted under
-access to copy from a designated place, then offering equivalent
+this License. No additional rights or licenses will be implied from the
-access to copy the source code from the same place counts as
+distribution or licensing of Covered Software under this License.
-distribution of the source code, even though third parties are not
+Notwithstanding Section 2.1(b) above, no patent license is granted by a
-compelled to copy the source along with the object code.
+Contributor:
 
-  4. You may not copy, modify, sublicense, or distribute the Program
+(a) for any code that a Contributor has removed from Covered Software;
-except as expressly provided under this License.  Any attempt
+    or
-otherwise to copy, modify, sublicense or distribute the Program is
-void, and will automatically terminate your rights under this License.
-However, parties who have received copies, or rights, from you under
-this License will not have their licenses terminated so long as such
-parties remain in full compliance.
 
-  5. You are not required to accept this License, since you have not
+(b) for infringements caused by: (i) Your and any other third party's
-signed it.  However, nothing else grants you permission to modify or
+    modifications of Covered Software, or (ii) the combination of its
-distribute the Program or its derivative works.  These actions are
+    Contributions with other software (except as part of its Contributor
-prohibited by law if you do not accept this License.  Therefore, by
+    Version); or
-modifying or distributing the Program (or any work based on the
-Program), you indicate your acceptance of this License to do so, and
-all its terms and conditions for copying, distributing or modifying
-the Program or works based on it.
 
-  6. Each time you redistribute the Program (or any work based on the
+(c) under Patent Claims infringed by Covered Software in the absence of
-Program), the recipient automatically receives a license from the
+    its Contributions.
-original licensor to copy, distribute or modify the Program subject to
-these terms and conditions.  You may not impose any further
-restrictions on the recipients' exercise of the rights granted herein.
-You are not responsible for enforcing compliance by third parties to
-this License.
 
-  7. If, as a consequence of a court judgment or allegation of patent
+This License does not grant any rights in the trademarks, service marks,
-infringement or for any other reason (not limited to patent issues),
+or logos of any Contributor (except as may be necessary to comply with
-conditions are imposed on you (whether by court order, agreement or
+the notice requirements in Section 3.4).
-otherwise) that contradict the conditions of this License, they do not
-excuse you from the conditions of this License.  If you cannot
-distribute so as to satisfy simultaneously your obligations under this
-License and any other pertinent obligations, then as a consequence you
-may not distribute the Program at all.  For example, if a patent
-license would not permit royalty-free redistribution of the Program by
-all those who receive copies directly or indirectly through you, then
-the only way you could satisfy both it and this License would be to
-refrain entirely from distribution of the Program.
 
-If any portion of this section is held invalid or unenforceable under
+2.4. Subsequent Licenses
-any particular circumstance, the balance of the section is intended to
-apply and the section as a whole is intended to apply in other
-circumstances.
 
-It is not the purpose of this section to induce you to infringe any
+No Contributor makes additional grants as a result of Your choice to
-patents or other property right claims or to contest validity of any
+distribute the Covered Software under a subsequent version of this
-such claims; this section has the sole purpose of protecting the
+License (see Section 10.2) or under the terms of a Secondary License (if
-integrity of the free software distribution system, which is
+permitted under the terms of Section 3.3).
-implemented by public license practices.  Many people have made
-generous contributions to the wide range of software distributed
-through that system in reliance on consistent application of that
-system; it is up to the author/donor to decide if he or she is willing
-to distribute software through any other system and a licensee cannot
-impose that choice.
 
-This section is intended to make thoroughly clear what is believed to
+2.5. Representation
-be a consequence of the rest of this License.
 
-  8. If the distribution and/or use of the Program is restricted in
+Each Contributor represents that the Contributor believes its
-certain countries either by patents or by copyrighted interfaces, the
+Contributions are its original creation(s) or it has sufficient rights
-original copyright holder who places the Program under this License
+to grant the rights to its Contributions conveyed by this License.
-may add an explicit geographical distribution limitation excluding
-those countries, so that distribution is permitted only in or among
-countries not thus excluded.  In such case, this License incorporates
-the limitation as if written in the body of this License.
 
-  9. The Free Software Foundation may publish revised and/or new versions
+2.6. Fair Use
-of the General Public License from time to time.  Such new versions will
-be similar in spirit to the present version, but may differ in detail to
-address new problems or concerns.
 
-Each version is given a distinguishing version number.  If the Program
+This License is not intended to limit any rights You have under
-specifies a version number of this License which applies to it and "any
+applicable copyright doctrines of fair use, fair dealing, or other
-later version", you have the option of following the terms and conditions
+equivalents.
-either of that version or of any later version published by the Free
-Software Foundation.  If the Program does not specify a version number of
-this License, you may choose any version ever published by the Free Software
-Foundation.
 
-  10. If you wish to incorporate parts of the Program into other free
+2.7. Conditions
-programs whose distribution conditions are different, write to the author
-to ask for permission.  For software which is copyrighted by the Free
-Software Foundation, write to the Free Software Foundation; we sometimes
-make exceptions for this.  Our decision will be guided by the two goals
-of preserving the free status of all derivatives of our free software and
-of promoting the sharing and reuse of software generally.
 
-			    NO WARRANTY
+Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted
+in Section 2.1.
 
-  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+3. Responsibilities
-FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
+-------------------
-OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
-PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
-OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
-MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
-TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
-PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
-REPAIR OR CORRECTION.
 
-  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+3.1. Distribution of Source Form
-WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
-REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
-INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
-OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
-TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
-YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
-PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
-POSSIBILITY OF SUCH DAMAGES.
 
-		     END OF TERMS AND CONDITIONS
+All distribution of Covered Software in Source Code Form, including any
+Modifications that You create or to which You contribute, must be under
+the terms of this License. You must inform recipients that the Source
+Code Form of the Covered Software is governed by the terms of this
+License, and how they can obtain a copy of this License. You may not
+attempt to alter or restrict the recipients' rights in the Source Code
+Form.
 
-	    How to Apply These Terms to Your New Programs
+3.2. Distribution of Executable Form
 
-  If you develop a new program, and you want it to be of the greatest
+If You distribute Covered Software in Executable Form then:
-possible use to the public, the best way to achieve this is to make it
-free software which everyone can redistribute and change under these terms.
 
-  To do so, attach the following notices to the program.  It is safest
+(a) such Covered Software must also be made available in Source Code
-to attach them to the start of each source file to most effectively
+    Form, as described in Section 3.1, and You must inform recipients of
-convey the exclusion of warranty; and each file should have at least
+    the Executable Form how they can obtain a copy of such Source Code
-the "copyright" line and a pointer to where the full notice is found.
+    Form by reasonable means in a timely manner, at a charge no more
+    than the cost of distribution to the recipient; and
 
-    <one line to give the program's name and a brief idea of what it does.>
+(b) You may distribute such Executable Form under the terms of this
-    Copyright (C) <year>  <name of author>
+    License, or sublicense it under different terms, provided that the
+    license for the Executable Form does not attempt to limit or alter
+    the recipients' rights in the Source Code Form under this License.
 
-    This program is free software; you can redistribute it and/or modify
+3.3. Distribution of a Larger Work
-    it under the terms of the GNU General Public License as published by
-    the Free Software Foundation; either version 2 of the License, or
-    (at your option) any later version.
 
-    This program is distributed in the hope that it will be useful,
+You may create and distribute a Larger Work under terms of Your choice,
-    but WITHOUT ANY WARRANTY; without even the implied warranty of
+provided that You also comply with the requirements of this License for
-    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+the Covered Software. If the Larger Work is a combination of Covered
-    GNU General Public License for more details.
+Software with a work governed by one or more Secondary Licenses, and the
+Covered Software is not Incompatible With Secondary Licenses, this
+License permits You to additionally distribute such Covered Software
+under the terms of such Secondary License(s), so that the recipient of
+the Larger Work may, at their option, further distribute the Covered
+Software under the terms of either this License or such Secondary
+License(s).
 
-    You should have received a copy of the GNU General Public License along
+3.4. Notices
-    with this program; if not, write to the Free Software Foundation, Inc.,
-    51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 
-Also add information on how to contact you by electronic and paper mail.
+You may not remove or alter the substance of any license notices
+(including copyright notices, patent notices, disclaimers of warranty,
+or limitations of liability) contained within the Source Code Form of
+the Covered Software, except that You may alter any license notices to
+the extent required to remedy known factual inaccuracies.
 
-If the program is interactive, make it output a short notice like this
+3.5. Application of Additional Terms
-when it starts in an interactive mode:
 
-    Gnomovision version 69, Copyright (C) year name of author
+You may choose to offer, and to charge a fee for, warranty, support,
-    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+indemnity or liability obligations to one or more recipients of Covered
-    This is free software, and you are welcome to redistribute it
+Software. However, You may do so only on Your own behalf, and not on
-    under certain conditions; type `show c' for details.
+behalf of any Contributor. You must make it absolutely clear that any
+such warranty, support, indemnity, or liability obligation is offered by
+You alone, and You hereby agree to indemnify every Contributor for any
+liability incurred by such Contributor as a result of warranty, support,
+indemnity or liability terms You offer. You may include additional
+disclaimers of warranty and limitations of liability specific to any
+jurisdiction.
 
-The hypothetical commands `show w' and `show c' should show the appropriate
+4. Inability to Comply Due to Statute or Regulation
-parts of the General Public License.  Of course, the commands you use may
+---------------------------------------------------
-be called something other than `show w' and `show c'; they could even be
-mouse-clicks or menu items--whatever suits your program.
 
-You should also get your employer (if you work as a programmer) or your
+If it is impossible for You to comply with any of the terms of this
-school, if any, to sign a "copyright disclaimer" for the program, if
+License with respect to some or all of the Covered Software due to
-necessary.  Here is a sample; alter the names:
+statute, judicial order, or regulation then You must: (a) comply with
+the terms of this License to the maximum extent possible; and (b)
+describe the limitations and the code they affect. Such description must
+be placed in a text file included with all distributions of the Covered
+Software under this License. Except to the extent prohibited by statute
+or regulation, such description must be sufficiently detailed for a
+recipient of ordinary skill to be able to understand it.
 
-  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+5. Termination
-  `Gnomovision' (which makes passes at compilers) written by James Hacker.
+--------------
 
-  <signature of Ty Coon>, 1 April 1989
+5.1. The rights granted under this License will terminate automatically
-  Ty Coon, President of Vice
+if You fail to comply with any of its terms. However, if You become
+compliant, then the rights granted under this License from a particular
+Contributor are reinstated (a) provisionally, unless and until such
+Contributor explicitly and finally terminates Your grants, and (b) on an
+ongoing basis, if such Contributor fails to notify You of the
+non-compliance by some reasonable means prior to 60 days after You have
+come back into compliance. Moreover, Your grants from a particular
+Contributor are reinstated on an ongoing basis if such Contributor
+notifies You of the non-compliance by some reasonable means, this is the
+first time You have received notice of non-compliance with this License
+from such Contributor, and You become compliant prior to 30 days after
+Your receipt of the notice.
 
-This General Public License does not permit incorporating your program into
+5.2. If You initiate litigation against any entity by asserting a patent
-proprietary programs.  If your program is a subroutine library, you may
+infringement claim (excluding declaratory judgment actions,
-consider it more useful to permit linking proprietary applications with the
+counter-claims, and cross-claims) alleging that a Contributor Version
-library.  If this is what you want to do, use the GNU Lesser General
+directly or indirectly infringes any patent, then the rights granted to
-Public License instead of this License.
+You by any and all Contributors for the Covered Software under Section
+2.1 of this License shall terminate.
+
+5.3. In the event of termination under Sections 5.1 or 5.2 above, all
+end user license agreements (excluding distributors and resellers) which
+have been validly granted by You or Your distributors under this License
+prior to termination shall survive termination.
+
+************************************************************************
+*                                                                      *
+*  6. Disclaimer of Warranty                                           *
+*  -------------------------                                           *
+*                                                                      *
+*  Covered Software is provided under this License on an "as is"       *
+*  basis, without warranty of any kind, either expressed, implied, or  *
+*  statutory, including, without limitation, warranties that the       *
+*  Covered Software is free of defects, merchantable, fit for a        *
+*  particular purpose or non-infringing. The entire risk as to the     *
+*  quality and performance of the Covered Software is with You.        *
+*  Should any Covered Software prove defective in any respect, You     *
+*  (not any Contributor) assume the cost of any necessary servicing,   *
+*  repair, or correction. This disclaimer of warranty constitutes an   *
+*  essential part of this License. No use of any Covered Software is   *
+*  authorized under this License except under this disclaimer.         *
+*                                                                      *
+************************************************************************
+
+************************************************************************
+*                                                                      *
+*  7. Limitation of Liability                                          *
+*  --------------------------                                          *
+*                                                                      *
+*  Under no circumstances and under no legal theory, whether tort      *
+*  (including negligence), contract, or otherwise, shall any           *
+*  Contributor, or anyone who distributes Covered Software as          *
+*  permitted above, be liable to You for any direct, indirect,         *
+*  special, incidental, or consequential damages of any character      *
+*  including, without limitation, damages for lost profits, loss of    *
+*  goodwill, work stoppage, computer failure or malfunction, or any    *
+*  and all other commercial damages or losses, even if such party      *
+*  shall have been informed of the possibility of such damages. This   *
+*  limitation of liability shall not apply to liability for death or   *
+*  personal injury resulting from such party's negligence to the       *
+*  extent applicable law prohibits such limitation. Some               *
+*  jurisdictions do not allow the exclusion or limitation of           *
+*  incidental or consequential damages, so this exclusion and          *
+*  limitation may not apply to You.                                    *
+*                                                                      *
+************************************************************************
+
+8. Litigation
+-------------
+
+Any litigation relating to this License may be brought only in the
+courts of a jurisdiction where the defendant maintains its principal
+place of business and such litigation shall be governed by laws of that
+jurisdiction, without reference to its conflict-of-law provisions.
+Nothing in this Section shall prevent a party's ability to bring
+cross-claims or counter-claims.
+
+9. Miscellaneous
+----------------
+
+This License represents the complete agreement concerning the subject
+matter hereof. If any provision of this License is held to be
+unenforceable, such provision shall be reformed only to the extent
+necessary to make it enforceable. Any law or regulation which provides
+that the language of a contract shall be construed against the drafter
+shall not be used to construe this License against a Contributor.
+
+10. Versions of the License
+---------------------------
+
+10.1. New Versions
+
+Mozilla Foundation is the license steward. Except as provided in Section
+10.3, no one other than the license steward has the right to modify or
+publish new versions of this License. Each version will be given a
+distinguishing version number.
+
+10.2. Effect of New Versions
+
+You may distribute the Covered Software under the terms of the version
+of the License under which You originally received the Covered Software,
+or under the terms of any subsequent version published by the license
+steward.
+
+10.3. Modified Versions
+
+If you create software not governed by this License, and you want to
+create a new license for such software, you may create and use a
+modified version of this License if you rename the license and remove
+any references to the name of the license steward (except to note that
+such modified license differs from this License).
+
+10.4. Distributing Source Code Form that is Incompatible With Secondary
+Licenses
+
+If You choose to distribute Source Code Form that is Incompatible With
+Secondary Licenses under the terms of this version of the License, the
+notice described in Exhibit B of this License must be attached.
+
+Exhibit A - Source Code Form License Notice
+-------------------------------------------
+
+  This Source Code Form is subject to the terms of the Mozilla Public
+  License, v. 2.0. If a copy of the MPL was not distributed with this
+  file, You can obtain one at https://mozilla.org/MPL/2.0/.
+
+If it is not possible or desirable to put the notice in a particular
+file, then You may include the notice in a location (such as a LICENSE
+file in a relevant directory) where a recipient would be likely to look
+for such a notice.
+
+You may add additional accurate notices of copyright ownership.
+
+Exhibit B - "Incompatible With Secondary Licenses" Notice
+---------------------------------------------------------
+
+  This Source Code Form is "Incompatible With Secondary Licenses", as
+  defined by the Mozilla Public License, v. 2.0.

MANIFEST.in

@@ -2,12 +2,8 @@ include LICENSE
 include README.rst
 recursive-include scripts *
 include atrcopy/templates/*
-include test_data/dos*atr
+include test_data/*_test?.atr
-include test_data/sd*atr
+include test_data/*_test?.atr.*
-include test_data/container*gz
-include test_data/container*bz2
-include test_data/container*xz
-include test_data/container*dcm
 include test_data/*.xex
 include test_data/rebuild.sh
 include test_data/create_binary.py

README.rst

@@ -22,11 +22,8 @@ Supported Python versions:
 
 * Python 3.6 (and later)
 
-If you need Python 2 support, ``atrcopy`` 6.5 and earlier supports:
+If you need Python 2 support, ``atrcopy`` 6.5 and earlier supports Python 2.7,
-
+which you can install with ``pip install "atrcopy<7.0"``
-* Python 2.7
-* Python 3.5
-* Python 3.6
 
 Dependencies
 ------------

atrcopy/__init__.py

@@ -6,7 +6,7 @@ import json
 import logging
 log = logging.getLogger(__name__)
 
-from ._metadata import __version__
+from ._version import __version__
 
 try:
     import numpy as np
@@ -19,10 +19,11 @@ from .dos33 import Dos33DiskImage
 from .kboot import KBootImage, add_xexboot_header
 from .segments import SegmentData, SegmentSaver, DefaultSegment, EmptySegment, ObjSegment, RawSectorsSegment, SegmentedFileSegment, user_bit_mask, match_bit_mask, comment_bit_mask, data_style, selected_bit_mask, diff_bit_mask, not_user_bit_mask, interleave_segments, SegmentList, get_style_mask, get_style_bits
 from .spartados import SpartaDosDiskImage
-from .cartridge import A8CartHeader, AtariCartImage
+from .cartridge import A8CartHeader, AtariCartImage, RomImage
-from .parsers import SegmentParser, DefaultSegmentParser, guess_parser_for_mime, guess_parser_for_system, guess_container, iter_parsers, iter_known_segment_parsers, mime_parse_order, parsers_for_filename
+from .parsers import SegmentParser, DefaultSegmentParser, guess_parser_by_size, guess_parser_for_mime, guess_parser_for_system, guess_container, iter_parsers, iter_known_segment_parsers, mime_parse_order, parsers_for_filename
 from .magic import guess_detail_for_mime
 from .utils import to_numpy, text_to_int
+from .dummy import LocalFilesystem
 
 
 def process(image, dirent, options):
@@ -53,34 +54,47 @@ def process(image, dirent, options):
         print(dirent)
 
 
-def find_diskimage(filename):
+def find_diskimage_from_data(data, verbose=False):
-    with open(filename, "rb") as fh:
+    data = to_numpy(data)
-        if options.verbose:
-            print("Loading file %s" % filename)
-        data = to_numpy(fh.read())
     parser = None
-    container = guess_container(data, options.verbose)
+    container = guess_container(data, verbose)
     if container is not None:
         data = container.unpacked
     rawdata = SegmentData(data)
-    for mime in mime_parse_order:
+    mime, parser = guess_parser_by_size(rawdata)
-        if options.verbose:
+    if parser is None:
-            print("Trying MIME type %s" % mime)
+        for mime in mime_parse_order:
-        parser = guess_parser_for_mime(mime, rawdata, options.verbose)
+            if verbose:
-        if parser is None:
+                print("Trying MIME type %s" % mime)
-            continue
+            parser = guess_parser_for_mime(mime, rawdata, verbose)
-        if options.verbose:
+            if parser is None:
-            print("Found parser %s" % parser.menu_name)
+                continue
-        mime2 = guess_detail_for_mime(mime, rawdata, parser)
+            if verbose:
-        if mime != mime2 and options.verbose:
+                print("Found parser %s" % parser.menu_name)
-            print("Signature match: %s" % mime2)
+            mime2 = guess_detail_for_mime(mime, rawdata, parser)
-        break
+            if mime != mime2:
+                mime = mime2
+                if verbose:
+                    print("Magic signature match: %s" % mime)
+            break
     if parser is None:
         raise errors.UnsupportedDiskImage("Unknown disk image type")
+    return parser, mime
+
+
+def find_diskimage(filename, verbose=False):
+    if filename == ".":
+        parser = LocalFilesystem()
+        mime = ""
     else:
-        parser.image.filename = filename
+        with open(filename, "rb") as fh:
-        parser.image.ext = ""
+            if verbose:
-        return parser
+                print("Loading file %s" % filename)
+            data = to_numpy(fh.read())
+        parser, mime = find_diskimage_from_data(data, verbose)
+    parser.image.filename = filename
+    parser.image.ext = ""
+    return parser, mime
 
 
 def extract_files(image, files):
@@ -95,12 +109,19 @@ def extract_files(image, files):
         output = dirent.filename
         if options.lower:
             output = output.lower()
+        if options.dir:
+            if not os.path.exists(options.dir):
+                os.makedirs(options.dir)
+            output = os.path.join(options.dir, output)
         if not options.dry_run:
             data = image.get_file(dirent)
             if os.path.exists(output) and not options.force:
                 print("skipping %s, file exists. Use -f to overwrite" % output)
                 continue
             print("extracting %s -> %s" % (name, output))
+            if options.text:
+                data = data.replace(b'\x7f', b'\t')
+                data = data.replace(b'\x9b', b'\n')
             with open(output, "wb") as fh:
                 fh.write(data)
         else:
@@ -132,6 +153,7 @@ def add_files(image, files):
     for name in files:
         with open(name, "rb") as fh:
             data = fh.read()
+        name = os.path.basename(name)
         changed = save_file(image, name, filetype, data)
     if changed:
         image.save()
@@ -223,13 +245,18 @@ def assemble_segments(source_files, data_files, obj_files, run_addr=""):
             log.debug("read data for %s" % s.name)
     for name in obj_files:
         try:
-            parser = find_diskimage(name)
+            parser, _ = find_diskimage(name, options.verbose)
         except errors.AtrError as e:
             print(f"skipping {name}: {e}")
         else:
             for s in parser.segments:
-                if s.origin > 0:
+                if hasattr(s, 'run_address'):
-                    print("adding %s from %s" % (s, name))
+                    if not run_addr:
+                        run_addr = s.run_address()
+                    else:
+                        print(f"already have run address {run_addr}; skipping {s.run_address()}")
+                elif s.origin > 0:
+                    print(f"adding {s} from {name}")
                     segments.add_segment(s.data, s.origin)
     if options.verbose:
         for s in segments:
@@ -237,14 +264,18 @@ def assemble_segments(source_files, data_files, obj_files, run_addr=""):
     if run_addr:
         try:
             run_addr = text_to_int(run_addr)
-        except ValueError:
+        except (AttributeError, ValueError):
-            run_addr = None
+            # not text, try as integer
+            try:
+                run_addr = int(run_addr)
+            except ValueError:
+                run_addr = None
 
     return segments, run_addr
 
 def assemble(image, source_files, data_files, obj_files, run_addr=""):
     segments, run_addr = assemble_segments(source_files, data_files, obj_files, run_addr)
-    file_data, filetype = image.create_executable_file_image(segments, run_addr)
+    file_data, filetype = image.create_executable_file_image(options.output, segments, run_addr)
     print("total file size: $%x (%d) bytes" % (len(file_data), len(file_data)))
     changed = save_file(image, options.output, filetype, file_data)
     if changed:
@@ -368,7 +399,7 @@ def create_image(template, name):
         else:
             with open(name, "wb") as fh:
                 fh.write(data)
-            parser = find_diskimage(name)
+            parser, _ = find_diskimage(name, options.verbose)
             print("created %s: %s" % (name, str(parser.image)))
             list_files(parser.image, [])
     else:
@@ -457,6 +488,8 @@ def run():
     extract_parser.add_argument("-l", "--lower", action="store_true", default=False, help="convert extracted filenames to lower case")
     #extract_parser.add_argument("-n", "--no-sys", action="store_true", default=False, help="only extract things that look like games (no DOS or .SYS files)")
     extract_parser.add_argument("-e", "--ext", action="store", nargs=1, default=False, help="add the specified extension")
+    extract_parser.add_argument("-d", "--dir", action="store", default=False, help="extract to the specified directory")
+    extract_parser.add_argument("-t", "--text", action="store_true", default=False, help="convert text files to unix-style text files")
     extract_parser.add_argument("-f", "--force", action="store_true", default=False, help="allow file overwrites on local filesystem")
     extract_parser.add_argument("files", metavar="FILENAME", nargs="*", help="if not using the -a/--all option, a file (or list of files) to extract from the disk image.")
 
@@ -574,12 +607,12 @@ def run():
         boot_image(disk_image_name, asm, data, obj, options.run_addr)
     else:
         try:
-            parser = find_diskimage(disk_image_name)
+            parser, mime = find_diskimage(disk_image_name, options.verbose)
         except (errors.UnsupportedContainer, errors.UnsupportedDiskImage, IOError) as e:
             print(f"{disk_image_name}: {e}")
         else:
             if command not in skip_diskimage_summary:
-                print("%s: %s" % (disk_image_name, parser.image))
+                print(f"{disk_image_name}: {parser.image}{' (%s}' % mime if mime and options.verbose else ''}")
             if command == "vtoc":
                 vtoc = parser.image.get_vtoc_object()
                 print(vtoc)

atrcopy/_metadata.py

@@ -1,4 +1,3 @@
-__version__ = "8.1"
 __author__ = "Rob McMullen"
 __author_email__ = "feedback@playermissile.com"
 __url__ = "https://github.com/robmcmullen/atrcopy"

atrcopy/_version.py

@@ -0,0 +1 @@
+__version__ = "10.1"

atrcopy/ataridos.py

@@ -1,9 +1,10 @@
 import numpy as np
 
 from . import errors
-from .diskimages import DiskImageBase, BaseHeader
+from .diskimages import DiskImageBase, BaseHeader, Bootable
 from .segments import SegmentData, EmptySegment, ObjSegment, RawSectorsSegment, DefaultSegment, SegmentedFileSegment, SegmentSaver, get_style_bits
 from .utils import *
+from .executables import get_xex
 
 import logging
 log = logging.getLogger(__name__)
@@ -256,7 +257,17 @@ class XexSegment(ObjSegment):
     savers = [SegmentSaver, XexSegmentSaver]
 
 
-class AtariDosFile:
+class RunAddressSegment(ObjSegment):
+    # FIXME: defining run_address as a property doesn't work for some reason.
+    # @property
+    # def run_address(self):
+    #     return self.rawdata[0:2].view(dtype="<u2")[0]
+    def run_address(self):
+        return self.rawdata[0:2].data.view(dtype="<u2")[0]
+
+
+
+class AtariDosFile(Bootable):
     """Parse a binary chunk into segments according to the Atari DOS object
     file format.
     
@@ -313,7 +324,11 @@ class AtariDosFile:
             if found < count:
                 self.segments.append(ObjSegment(r[pos + 4:pos + 4 + count], pos, pos + 4, start, end, "Incomplete Data"))
                 break
-            self.segments.append(ObjSegment(r[pos + 4:pos + 4 + count], pos, pos + 4, start, end))
+            if start == 0x2e0:
+                segment_cls = RunAddressSegment
+            else:
+                segment_cls = ObjSegment
+            self.segments.append(segment_cls(r[pos + 4:pos + 4 + count], pos, pos + 4, start, end))
             pos += 4 + count
             style_pos = pos
 
@@ -438,6 +453,8 @@ class XfdHeader(AtrHeader):
 
 
 class AtariDosDiskImage(DiskImageBase):
+    default_executable_extension = "XEX"
+
     def __init__(self, *args, **kwargs):
         self.first_vtoc = 360
         self.num_vtoc = 1
@@ -661,10 +678,6 @@ class AtariDosDiskImage(DiskImageBase):
                 log.debug("%s not a binary file; skipping segment generation" % str(segment))
         return segments_out
 
-    def create_executable_file_image(self, segments, run_addr=None):
-        base_segment, user_segments = get_xex(segments, run_addr)
-        return base_segment.data, "XEX"
-
 
 class BootDiskImage(AtariDosDiskImage):
     def __str__(self):
@@ -744,56 +757,6 @@ class AtariDiskImage(BootDiskImage):
         return []
 
 
-def get_xex(segments, run_addr=None):
-    segments_copy = [s for s in segments]  # don't affect the original list!
-    main_segment = None
-    sub_segments = []
-    data_style = get_style_bits(data=True)
-    total = 2
-    runad = False
-    for s in segments:
-        total += 4 + len(s)
-        if s.origin == 0x2e0:
-            runad = True
-    if not runad:
-        words = np.empty([1], dtype='<u2')
-        if run_addr:
-            found = False
-            for s in segments:
-                if run_addr >= s.origin and run_addr < s.origin + len(s):
-                    found = True
-                    break
-            if not found:
-                raise errors.InvalidBinaryFile("Run address points outside data segments")
-        else:
-            run_addr = segments[0].origin
-        words[0] = run_addr
-        r = SegmentData(words.view(dtype=np.uint8))
-        s = DefaultSegment(r, 0x2e0)
-        segments_copy[0:0] = [s]
-        total += 6
-    bytes = np.zeros([total], dtype=np.uint8)
-    rawdata = SegmentData(bytes)
-    main_segment = DefaultSegment(rawdata)
-    main_segment.data[0:2] = 0xff  # FFFF header
-    main_segment.style[0:2] = data_style
-    i = 2
-    for s in segments_copy:
-        # create new sub-segment inside new main segment that duplicates the
-        # original segment's data/style
-        new_s = DefaultSegment(rawdata[i:i+4+len(s)], s.origin)
-        words = new_s.data[0:4].view(dtype='<u2')
-        words[0] = s.origin
-        words[1] = s.origin + len(s) - 1
-        new_s.style[0:4] = data_style
-        new_s.data[4:4+len(s)] = s[:]
-        new_s.style[4:4+len(s)] = s.style[:]
-        i += 4 + len(s)
-        new_s.copy_user_data(s, 4)
-        sub_segments.append(new_s)
-    return main_segment, sub_segments
-
-
 def add_atr_header(bytes):
     header = AtrHeader(create=True)
     header.check_size(len(bytes))

atrcopy/cartridge.py

@@ -119,6 +119,7 @@ class A8CartHeader:
         ('checksum', '>u4'),
         ('unused','>u4')
         ])
+    nominal_length = format.itemsize
     file_format = "Cart"
 
     def __init__(self, bytes=None, create=False):
@@ -138,7 +139,7 @@ class A8CartHeader:
         self.main_origin = 0
         self.possible_types = set()
         if create:
-            self.header_offset = 16
+            self.header_offset = self.nominal_length
             self.check_size(0)
         if bytes is None:
             return
@@ -149,7 +150,7 @@ class A8CartHeader:
                 raise errors.InvalidCartHeader
             self.cart_type = int(values[1])
             self.crc = int(values[2])
-            self.header_offset = 16
+            self.header_offset = self.nominal_length
             self.set_type(self.cart_type)
         else:
             raise errors.InvalidCartHeader
@@ -160,8 +161,15 @@ class A8CartHeader:
     def __len__(self):
         return self.header_offset
 
+    @property
+    def valid(self):
+        return self.cart_type != -1
+
+    def calc_crc_from_data(self, data):
+        self.crc = 0
+
     def to_array(self):
-        raw = np.zeros([16], dtype=np.uint8)
+        raw = np.zeros([self.nominal_length], dtype=np.uint8)
         values = raw.view(dtype=self.format)[0]
         values[0] = b'CART'
         values[1] = self.cart_type
@@ -189,18 +197,14 @@ class A8CartHeader:
     def check_size(self, size):
         self.possible_types = set()
         k, r = divmod(size, 1024)
-        if r == 0 or r == 16:
+        if r == 0 or r == self.nominal_length:
             for i, t in enumerate(known_cart_types):
                 valid_size = t[0]
                 if k == valid_size:
                     self.possible_types.add(i)
 
 
-class AtariCartImage(DiskImageBase):
+class BaseAtariCartImage(DiskImageBase):
-    def __init__(self, rawdata, cart_type, filename=""):
-        self.cart_type = cart_type
-        DiskImageBase.__init__(self, rawdata, filename)
-
     def __str__(self):
         return str(self.header)
 
@@ -210,23 +214,22 @@ class AtariCartImage(DiskImageBase):
             self.header = A8CartHeader(data)
         except errors.InvalidCartHeader:
             self.header = A8CartHeader()
-            self.header.set_type(self.cart_type)
 
     def strict_check(self):
-        if self.header.cart_type != self.cart_type:
+        raise NotImplementedError
-            raise errors.InvalidDiskImage("Cart type doesn't match type defined in header")
 
     def relaxed_check(self):
         if self.header.cart_type != self.cart_type:
             # force the header to be the specified cart type
             self.header = A8CartHeader()
             self.header.set_type(self.cart_type)
+            self.check_size()
 
     def check_size(self):
-        if self.header is None:
+        if not self.header.valid:
             return
         k, rem = divmod((len(self) - len(self.header)), 1024)
-        c = get_cart(self.cart_type)
+        c = get_cart(self.header.cart_type)
         log.debug("checking type=%d, k=%d, rem=%d for %s, %s" % (self.cart_type, k, rem, c[1], c[2]))
         if rem > 0:
             raise errors.InvalidDiskImage("Cart not multiple of 1K")
@@ -258,6 +261,47 @@ class AtariCartImage(DiskImageBase):
             segments.append(s)
         return segments
 
+    def create_emulator_boot_segment(self):
+        h = self.header
+        k, rem = divmod(len(self), 1024)
+        if rem == 0:
+            h.calc_crc_from_data(self.bytes)
+            data_with_header = np.empty(len(self) + h.nominal_length, dtype=np.uint8)
+            data_with_header[0:h.nominal_length] = h.to_array()
+            data_with_header[h.nominal_length:] = self.bytes
+            r = SegmentData(data_with_header)
+        else:
+            r = self.rawdata
+        s = ObjSegment(r, 0, 0, self.header.main_origin, name="Cart image")
+        return s
+
+
+class AtariCartImage(BaseAtariCartImage):
+    def __init__(self, rawdata, cart_type, filename=""):
+        c = get_cart(cart_type)
+        self.cart_type = cart_type
+        DiskImageBase.__init__(self, rawdata, filename)
+
+    def strict_check(self):
+        if not self.header.valid:
+            raise errors.InvalidDiskImage("Missing cart header")
+        if self.header.cart_type != self.cart_type:
+            raise errors.InvalidDiskImage("Cart type doesn't match type defined in header")
+
+
+class Atari8bitCartImage(AtariCartImage):
+    def strict_check(self):
+        if "5200" in self.header.cart_name:
+            raise errors.InvalidDiskImage("5200 Carts don't work in the home computers.")
+        AtariCartImage.strict_check(self)
+
+
+class Atari5200CartImage(AtariCartImage):
+    def strict_check(self):
+        if "5200" not in self.header.cart_name:
+            raise errors.InvalidDiskImage("Home computer carts don't work in the 5200.")
+        AtariCartImage.strict_check(self)
+
 
 def add_cart_header(bytes):
     header = A8CartHeader(create=True)
@@ -267,3 +311,45 @@ def add_cart_header(bytes):
     data[0:hlen] = header.to_array()
     data[hlen:] = bytes
     return data
+
+
+class RomImage(DiskImageBase):
+    def __str__(self):
+        return f"{len(self.rawdata) // 1024}k ROM image"
+
+    def read_header(self):
+        self.header = A8CartHeader()
+
+    def strict_check(self):
+        self.check_size()
+
+    def check_size(self):
+        size = len(self)
+        if (size & (size - 1)) != 0:
+            raise errors.InvalidDiskImage("ROM image not a power of 2")
+
+    def parse_segments(self):
+        r = self.rawdata
+        s = ObjSegment(r, 0, 0, self.header.main_origin, name="Main Bank")
+        self.segments = [s]
+
+    def create_emulator_boot_segment(self):
+        s = self.segments[0]
+        if s.origin == 0:
+            return None
+        return s
+
+
+class Atari2600CartImage(RomImage):
+    def __str__(self):
+        return f"{len(self.rawdata) // 1024}k Atari 2600 Cartridge"
+
+
+class Atari2600StarpathImage(RomImage):
+    def __str__(self):
+        return f"{len(self.rawdata) // 1024}k Atari 2600 Starpath Cassette"
+
+
+class VectrexCartImage(RomImage):
+    def __str__(self):
+        return f"{len(self.rawdata) // 1024}k Vectrex Cartridge"

atrcopy/container.py

@@ -23,7 +23,10 @@ class DiskImageContainer:
 
     def __unpack_raw_data(self, data):
         raw = data.tobytes()
-        unpacked = self.unpack_bytes(raw)
+        try:
+            unpacked = self.unpack_bytes(raw)
+        except EOFError as e:
+            raise errors.InvalidContainer(e)
         return to_numpy(unpacked)
 
     def unpack_bytes(self, byte_data):

atrcopy/diskimages.py

@@ -3,6 +3,7 @@ import numpy as np
 from . import errors
 from .segments import SegmentData, EmptySegment, ObjSegment, RawSectorsSegment
 from .utils import *
+from .executables import create_executable_file_data
 
 import logging
 log = logging.getLogger(__name__)
@@ -91,7 +92,14 @@ class BaseHeader:
         return self.sector_class(self.sector_size, data)
 
 
-class DiskImageBase:
+class Bootable:
+    def create_emulator_boot_segment(self):
+        return ObjSegment(self.rawdata, 0, 0, 0)
+
+
+class DiskImageBase(Bootable):
+    default_executable_extension = None
+
     def __init__(self, rawdata, filename="", create=False):
         self.rawdata = rawdata
         self.bytes = self.rawdata.get_data()
@@ -188,7 +196,7 @@ class DiskImageBase:
 
     def assert_valid_sector(self, sector):
         if not self.header.sector_is_valid(sector):
-            raise ByteNotInFile166("Sector %d out of range" % sector)
+            raise errors.ByteNotInFile166("Sector %d out of range" % sector)
 
     def check_sane(self):
         if not self.all_sane:
@@ -300,8 +308,12 @@ class DiskImageBase:
             segments.append(segment)
         return segments
 
-    def create_executable_file_image(self, segments, run_addr=None):
+    def create_executable_file_image(self, output_name, segments, run_addr=None):
-        raise errors.NotImplementedError
+        try:
+            data, filetype = create_executable_file_data(output_name, segments, run_addr)
+        except errors.UnsupportedContainer:
+            data, filetype = create_executable_file_data(self.default_executable_extension, segments, run_addr)
+        return data, filetype
 
     @classmethod
     def create_boot_image(self, segments, run_addr=None):

atrcopy/dos33.py

@@ -1,9 +1,10 @@
 import numpy as np
 
 from . import errors
-from .diskimages import BaseHeader, DiskImageBase
+from .diskimages import BaseHeader, DiskImageBase, Bootable
 from .utils import Directory, VTOC, WriteableSector, BaseSectorList, Dirent
 from .segments import DefaultSegment, EmptySegment, ObjSegment, RawTrackSectorSegment, SegmentSaver, get_style_bits, SegmentData
+from .executables import get_bsave
 
 import logging
 log = logging.getLogger(__name__)
@@ -250,7 +251,7 @@ class Dos33Dirent(Dirent):
         values[1] = self.sector
         values[2] = self.flag
         n = min(len(self.filename), 30)
-        data[3:3+n] = np.fromstring(self.filename.encode("ascii"), dtype=np.uint8) | 0x80
+        data[3:3+n] = np.frombuffer(self.filename.encode("ascii"), dtype=np.uint8) | 0x80
         data[3+n:] = ord(' ') | 0x80
         if self.deleted:
             data[0x20] = self.track
@@ -381,6 +382,8 @@ class Dos33Header(BaseHeader):
 
 
 class Dos33DiskImage(DiskImageBase):
+    default_executable_extension = "BSAVE"
+
     def __init__(self, rawdata, filename=""):
         DiskImageBase.__init__(self, rawdata, filename)
         self.default_filetype = "B"
@@ -444,6 +447,9 @@ class Dos33DiskImage(DiskImageBase):
         values = data[0:self.vtoc_type.itemsize].view(dtype=self.vtoc_type)[0]
         self.header.first_directory = self.header.sector_from_track(values['cat_track'], values['cat_sector'])
         self.header.sector_size = int(values['sector_size'])
+        if self.header.sector_size != 256:
+            log.warning(f"Nonstandard sector size {self.header.sector_size}; this is likely an error, setting to 256")
+            self.header.sector_size = 256
         self.header.max_sectors = int(values['num_tracks']) * int(values['sectors_per_track'])
         self.header.ts_pairs = int(values['max_pairs'])
         self.header.dos_release = values['dos_release']
@@ -590,46 +596,8 @@ class Dos33DiskImage(DiskImageBase):
             segment = EmptySegment(self.rawdata, name=dirent.filename)
         return segment
 
-    def create_executable_file_image(self, segments, run_addr=None):
-        # Apple 2 executables get executed at the first address loaded. If the
-        # run_addr is not the first byte of the combined data, have to create a
-        # new 3-byte segment with a "JMP run_addr" to go at the beginning
-        origin = 100000000
-        last = -1
 
-        for s in segments:
+class Dos33BinFile(Bootable):
-            origin = min(origin, s.origin)
-            last = max(last, s.origin + len(s))
-            if _xd: log.debug("contiguous bytes needed: %04x - %04x" % (origin, last))
-        if run_addr and run_addr != origin:
-            # check if run_addr points to some location that has data
-            found = False
-            for s in segments:
-                if run_addr >= s.origin and run_addr < s.origin + len(s):
-                    found = True
-                    break
-            if not found:
-                raise errors.InvalidBinaryFile("Run address points outside data segments")
-            origin -= 3
-            hi, lo = divmod(run_addr, 256)
-            raw = SegmentData([0x4c, lo, hi])
-            all_segments = [DefaultSegment(raw, origin=origin)]
-            all_segments.extend(segments)
-        else:
-            all_segments = segments
-        size = last - origin
-        image = np.zeros([size + 4], dtype=np.uint8)
-        words = image[0:4].view(dtype="<u2")  # always little endian
-        words[0] = origin
-        words[1] = size
-        for s in all_segments:
-            index = s.origin - origin + 4
-            print("setting data for $%04x - $%04x at index $%04x" % (s.origin, s.origin + len(s), index))
-            image[index:index + len(s)] = s.data
-        return image, 'B'
-
-
-class Dos33BinFile:
     """Parse a binary chunk into segments according to the DOS 3.3 binary
     dump format
     """
@@ -672,6 +640,9 @@ class Dos33BinFile:
         else:
             raise errors.InvalidBinaryFile(f"Invalid BSAVE header")
 
+    def create_emulator_boot_segment(self):
+        return self.segments[0]
+
 
 class ProdosHeader(Dos33Header):
     file_format = "ProDOS"

atrcopy/dummy.py

@@ -0,0 +1,44 @@
+import os
+
+import numpy as np
+
+from . import errors
+from .segments import SegmentData, EmptySegment, ObjSegment
+from .diskimages import DiskImageBase
+from .utils import to_numpy
+
+import logging
+log = logging.getLogger(__name__)
+
+
+class LocalFilesystemImage(DiskImageBase):
+    def __init__(self, path):
+        self.path = path
+
+    def __str__(self, path="."):
+        return f"Local filesystem output to: {self.path}"
+
+    def save(self, filename=""):
+        # This is to save the disk image containing the files on the disk image
+        # to the local disk, which doesn't make sense when the disk image is
+        # the filesystem.
+        pass
+
+    def find_dirent(self, name):
+        path = os.path.join(self.path, name)
+        if os.path.exists(path):
+            return True
+        raise errors.FileNotFound("%s not found on disk" % str(name))
+
+    def write_file(self, name, filetype, data):
+        path = os.path.join(self.path, name)
+        with open(path, "wb") as fh:
+            fh.write(data)
+
+    def delete_file(self, name):
+        pass
+
+
+class LocalFilesystem():
+    def __init__(self, path="."):
+        self.image = LocalFilesystemImage(path)

atrcopy/executables.py

@@ -0,0 +1,112 @@
+import numpy as np
+
+from . import errors
+from .segments import SegmentData, EmptySegment, ObjSegment, RawSectorsSegment, DefaultSegment, SegmentedFileSegment, SegmentSaver, get_style_bits
+from .utils import *
+
+import logging
+log = logging.getLogger(__name__)
+try:  # Expensive debugging
+    _xd = _expensive_debugging
+except NameError:
+    _xd = False
+
+
+def get_xex(segments, run_addr=None):
+    segments_copy = [s for s in segments]  # don't affect the original list!
+    main_segment = None
+    sub_segments = []
+    data_style = get_style_bits(data=True)
+    total = 2
+    runad = False
+    for s in segments:
+        total += 4 + len(s)
+        if s.origin == 0x2e0:
+            runad = True
+    if not runad:
+        words = np.empty([1], dtype='<u2')
+        if run_addr:
+            found = False
+            for s in segments:
+                if run_addr >= s.origin and run_addr < s.origin + len(s):
+                    found = True
+                    break
+            if not found:
+                raise errors.InvalidBinaryFile("Run address points outside data segments")
+        else:
+            run_addr = segments[0].origin
+        words[0] = run_addr
+        r = SegmentData(words.view(dtype=np.uint8))
+        s = DefaultSegment(r, 0x2e0)
+        segments_copy[0:0] = [s]
+        total += 6
+    bytes = np.zeros([total], dtype=np.uint8)
+    rawdata = SegmentData(bytes)
+    main_segment = DefaultSegment(rawdata)
+    main_segment.data[0:2] = 0xff  # FFFF header
+    main_segment.style[0:2] = data_style
+    i = 2
+    for s in segments_copy:
+        # create new sub-segment inside new main segment that duplicates the
+        # original segment's data/style
+        new_s = DefaultSegment(rawdata[i:i+4+len(s)], s.origin)
+        words = new_s.data[0:4].view(dtype='<u2')
+        words[0] = s.origin
+        words[1] = s.origin + len(s) - 1
+        new_s.style[0:4] = data_style
+        new_s.data[4:4+len(s)] = s[:]
+        new_s.style[4:4+len(s)] = s.style[:]
+        i += 4 + len(s)
+        new_s.copy_user_data(s, 4)
+        sub_segments.append(new_s)
+    return main_segment, sub_segments
+
+
+def get_bsave(segments, run_addr=None):
+    # Apple 2 executables get executed at the first address loaded. If the
+    # run_addr is not the first byte of the combined data, have to create a
+    # new 3-byte segment with a "JMP run_addr" to go at the beginning
+    origin = 100000000
+    last = -1
+
+    for s in segments:
+        origin = min(origin, s.origin)
+        last = max(last, s.origin + len(s))
+        if _xd: log.debug("contiguous bytes needed: %04x - %04x" % (origin, last))
+    if run_addr and run_addr != origin:
+        # check if run_addr points to some location that has data
+        found = False
+        for s in segments:
+            if run_addr >= s.origin and run_addr < s.origin + len(s):
+                found = True
+                break
+        if not found:
+            raise errors.InvalidBinaryFile("Run address points outside data segments")
+        origin -= 3
+        hi, lo = divmod(run_addr, 256)
+        raw = SegmentData([0x4c, lo, hi])
+        all_segments = [DefaultSegment(raw, origin=origin)]
+        all_segments.extend(segments)
+    else:
+        all_segments = segments
+    size = last - origin
+    image = np.zeros([size + 4], dtype=np.uint8)
+    words = image[0:4].view(dtype="<u2")  # always little endian
+    words[0] = origin
+    words[1] = size
+    for s in all_segments:
+        index = s.origin - origin + 4
+        log.debug("setting data for $%04x - $%04x at index $%04x" % (s.origin, s.origin + len(s), index))
+        image[index:index + len(s)] = s.data
+    return image
+
+
+def create_executable_file_data(filename, segments, run_addr=None):
+    name = filename.lower()
+    if name.endswith("xex"):
+        base_segment, user_segments = get_xex(segments, run_addr)
+        return base_segment.data, "XEX"
+    elif name.endswith("bin") or name.endswith("bsave"):
+        data = get_bsave(segments, run_addr)
+        return data, "B"
+    raise errors.UnsupportedContainer

atrcopy/kboot.py

@@ -1,7 +1,7 @@
 import numpy as np
 
 from . import errors
-from .ataridos import AtariDosDirent, AtariDosDiskImage, XexSegment, get_xex
+from .ataridos import AtrHeader, AtariDosDirent, AtariDosDiskImage, XexSegment, get_xex
 from .segments import SegmentData
 
 
@@ -12,11 +12,11 @@ class KBootDirent(AtariDosDirent):
         self.starting_sector = 4
         self.basename = image.filename
         if not self.basename:
-            self.basename = "KBOOT"
+            self.basename = b"KBOOT"
         if self.basename == self.basename.upper():
-            self.ext = "XEX"
+            self.ext = b"XEX"
         else:
-            self.ext = "xex"
+            self.ext = b"xex"
         start, size = image.header.get_pos(4)
         i = image.header.header_offset + 9
         count = image.bytes[i] + 256 * image.bytes[i+1] + 256 * 256 *image.bytes[i + 2]
@@ -70,7 +70,13 @@ class KBootImage(AtariDosDiskImage):
     @classmethod
     def create_boot_image(cls, segments, run_addr=None):
         data_segment, _ = get_xex(segments)
-        data_bytes = add_xexboot_header(data_segment.data)
+        payload_bytes = add_xexboot_header(data_segment.data)
+        data_bytes = np.zeros(len(payload_bytes) + 16, np.uint8)
+        data_bytes[16:] = payload_bytes[:]
+        header_bytes = data_bytes[0:16]
+        atr_header = AtrHeader(create=True)
+        atr_header.check_size(len(payload_bytes))
+        atr_header.encode(header_bytes)
         raw = SegmentData(data_bytes)
         atr = cls(raw, create=True)
         return atr
@@ -80,7 +86,7 @@ xexboot_header = b'\x00\x03\x00\x07\r\x07L\r\x07\x1c[\x00\x00\xa0\x00\x8c\t\x03\
 
 
 def insert_bytes(data, offset, string, color):
-    s = np.fromstring(string.upper(), dtype=np.uint8) - 32  # convert to internal
+    s = np.frombuffer(string.upper(), dtype=np.uint8) - 32  # convert to internal
     s = s | color
     count = len(s)
     tx = offset + (20 - count) // 2
@@ -97,7 +103,7 @@ def add_xexboot_header(bytes, bootcode=None, title=b"DEMO", author=b"an atari us
     paragraphs = padded_size // 16
 
     if bootcode is None:
-        bootcode = np.fromstring(xexboot_header, dtype=np.uint8)
+        bootcode = np.copy(np.frombuffer(xexboot_header, dtype=np.uint8))
     else:
         # don't insert title or author in user supplied bootcode; would have to
         # assume that the user supplied everything desired in their own code!

atrcopy/magic.py

@@ -65,9 +65,9 @@ def check_signature(raw, sig):
 def guess_detail_for_mime(mime, raw, parser):
     for entry in magic:
         if entry['mime'].startswith(mime):
-            log.debug("checking signature for %s" % entry['mime'])
+            log.debug("checking entry for %s" % entry['mime'])
             if check_signature(raw, entry['signature']):
-                log.debug("found signature: %s" % entry['name'])
+                log.debug("found match: %s" % entry['name'])
                 return entry['mime']
     return mime
 

atrcopy/mame.py

@@ -46,7 +46,7 @@ class MameZipImage(DiskImageBase):
         segment_info = []
         offset = 0
         for item in zf.infolist():
-            rom = np.fromstring(zf.open(item).read(), dtype=np.uint8)
+            rom = np.frombuffer(zf.open(item).read(), dtype=np.uint8)
             roms.append(rom)
             segment_info.append((offset, item.file_size, item.filename, item.CRC))
             offset += item.file_size

atrcopy/omnivore_loader.py

@@ -0,0 +1,27 @@
+from . import find_diskimage_from_data, errors
+
+import logging
+log = logging.getLogger(__name__)
+
+
+def identify_mime(header, fh):
+    mime_type = None
+    try:
+        fh.seek(0)
+        data = fh.read()
+    except IOError as e:
+        log.debug(f"atrcopy loader: error reading entire file: {e}")
+    else:
+        try:
+            parser, mime_type = find_diskimage_from_data(data, True)
+        except (errors.UnsupportedContainer, errors.UnsupportedDiskImage, IOError) as e:
+            log.debug(f"error in atrcopy parser: {e}")
+        else:
+            log.debug(f"{parser.image}: {mime_type}")
+
+        if mime_type:
+            log.debug(f"atrcopy loader: identified {mime_type}")
+            return dict(mime=mime_type, ext="", atrcopy_parser=parser)
+        else:
+            log.debug(f"atrcopy loader: not recognized")
+    return None

atrcopy/parsers.py

@@ -1,10 +1,12 @@
+import hashlib
+
 import numpy as np
 
 from .segments import SegmentData, DefaultSegment
 from .kboot import KBootImage
 from .ataridos import AtariDosDiskImage, BootDiskImage, AtariDosFile, XexContainerSegment, AtariDiskImage
 from .spartados import SpartaDosDiskImage
-from .cartridge import AtariCartImage, get_known_carts
+from .cartridge import AtariCartImage, Atari8bitCartImage, Atari5200CartImage, get_known_carts, RomImage, Atari2600CartImage, Atari2600StarpathImage, VectrexCartImage
 from .mame import MameZipImage
 from .dos33 import Dos33DiskImage, ProdosDiskImage, Dos33BinFile
 from .standard_delivery import StandardDeliveryImage
@@ -12,6 +14,7 @@ from . import errors
 from .magic import guess_detail_for_mime
 from . import container
 from .dcm import DCMContainer
+from .signatures import sha1_signatures
 
 import logging
 log = logging.getLogger(__name__)
@@ -69,6 +72,7 @@ class SegmentParser:
         self.segments.append(self.container_segment(r, 0, name=self.menu_name))
         try:
             log.debug("Trying %s" % self.image_type)
+            log.debug(self.image_type.__mro__)
             self.image = self.get_image(r)
             self.check_image()
             self.image.parse_segments()
@@ -78,7 +82,14 @@ class SegmentParser:
             raise errors.InvalidSegmentParser(e)
         self.segments.extend(self.image.segments)
 
+    def reconstruct_segments(self, new_rawdata):
+        self.image = self.get_image(new_rawdata)
+        self.segment_data = new_rawdata
+        for s in self.segments:
+            s.reconstruct_raw(new_rawdata)
+
     def get_image(self, r):
+        log.info(f"checking image type {self.image_type}")
         return self.image_type(r)
 
     def check_image(self):
@@ -136,9 +147,40 @@ class AtariCartSegmentParser(SegmentParser):
     cart_info = None
 
     def get_image(self, r):
+        log.info(f"checking cart type {self.cart_type}: {self.image_type}")
         return self.image_type(r, self.cart_type)
 
 
+class Atari8bitCartParser(AtariCartSegmentParser):
+    menu_name = "Atari Home Computer Cartridge"
+    image_type = Atari8bitCartImage
+
+
+class Atari5200CartParser(AtariCartSegmentParser):
+    menu_name = "Atari 5200 Cartridge"
+    image_type = Atari5200CartImage
+
+
+class Atari2600CartParser(SegmentParser):
+    menu_name = "Atari 2600 Cartridge"
+    image_type = Atari2600CartImage
+
+
+class Atari2600StarpathParser(SegmentParser):
+    menu_name = "Atari 2600 Starpath Cassette"
+    image_type = Atari2600StarpathImage
+
+
+class VectrexParser(SegmentParser):
+    menu_name = "Vectrex Cartridge"
+    image_type = VectrexCartImage
+
+
+class RomParser(SegmentParser):
+    menu_name = "ROM Image"
+    image_type = RomImage
+
+
 class MameZipParser(SegmentParser):
     menu_name = "MAME ROM Zipfile"
     image_type = MameZipImage
@@ -179,9 +221,39 @@ def guess_container(r, verbose=False):
             if verbose:
                 log.info(f"found container {c}")
             return found
+    log.info(f"image does not appear to be a container.")
     return None
 
 
+def guess_parser_by_size(r, verbose=False):
+    found = None
+    mime = None
+    size = len(r)
+    if size in sha1_signatures:
+        sha_hash = hashlib.sha1(r.data).digest()
+        log.info(f"{size} in signature database, attempting to match {sha_hash}")
+        try:
+            match = sha1_signatures[size][sha_hash]
+        except KeyError:
+            pass
+        else:
+            mime = match[0]
+            log.info(f"found match: {match}")
+            parsers = mime_parsers[mime]
+            for parser in parsers:
+                try:
+                    found = parser(r, False)
+                    break
+                except errors.InvalidSegmentParser as e:
+                    if verbose:
+                        log.info("parser isn't %s: %s" % (parser.__name__, str(e)))
+                    pass
+        if found is None:
+            log.info(f"no matching signature")
+    else:
+        log.info(f"{size} not found in signature database; skipping sha1 matching")
+    return mime, found
+
 def guess_parser_for_mime(mime, r, verbose=False):
     parsers = mime_parsers[mime]
     found = None
@@ -210,12 +282,15 @@ def iter_parsers(r):
     container = guess_container(r.data)
     if container is not None:
         r = SegmentData(container.unpacked)
-    for mime in mime_parse_order:
+    mime, parser = guess_parser_by_size(r)
-        p = guess_parser_for_mime(mime, r)
+    if parser is None:
-        if p is not None:
+        for mime in mime_parse_order:
-            mime = guess_detail_for_mime(mime, r, p)
+            p = guess_parser_for_mime(mime, r)
-            return mime, p
+            if p is not None:
-    return None, None
+                parser = p
+                mime = guess_detail_for_mime(mime, r, p)
+                break
+    return mime, parser
 
 
 def parsers_for_filename(name):
@@ -251,12 +326,26 @@ mime_parsers = {
         XexSegmentParser,
         ],
     "application/vnd.atari8bit.cart": [
+        Atari8bitCartParser,
         ],
-    "application/vnd.atari8bit.5200_cart": [
+    "application/vnd.atari5200.cart": [
+        Atari5200CartParser,
+        ],
+    "application/vnd.atari2600.cart": [
+        Atari2600CartParser,
+        ],
+    "application/vnd.atari2600.starpath": [
+        Atari2600StarpathParser,
+        ],
+    "application/vnd.vectrex": [
+        VectrexParser,
         ],
     "application/vnd.mame_rom": [
         MameZipParser,
         ],
+    "application/vnd.rom": [
+        RomParser,
+        ],
     "application/vnd.apple2.dsk": [
         Dos33SegmentParser,
         ProdosSegmentParser,
@@ -266,22 +355,47 @@ mime_parsers = {
         ],
     }
 
+# Note: Atari 2600 scanning not performed here because it will match everything
 mime_parse_order = [
     "application/vnd.atari8bit.atr",
     "application/vnd.atari8bit.xex",
     "application/vnd.atari8bit.cart",
-    "application/vnd.atari8bit.5200_cart",
+    "application/vnd.atari5200.cart",
     "application/vnd.mame_rom",
     "application/vnd.apple2.dsk",
     "application/vnd.apple2.bin",
+    "application/vnd.rom",
+    ]
+
+# different than the above mime_parse_order, this list is the order in which
+# the mime parsers will appear in a UI. Some, like the vectrex and atari2600
+# parsers, aren't included in the parse order because they will identify
+# many things incorrectly. They are used only when parsing by size and
+# signature.
+mime_display_order = [
+    "application/vnd.atari8bit.atr",
+    "application/vnd.atari8bit.xex",
+    "application/vnd.atari8bit.cart",
+    "application/vnd.atari5200.cart",
+    "application/vnd.atari2600.cart",
+    "application/vnd.atari2600.starpath",
+    "application/vnd.vectrex",
+    "application/vnd.mame_rom",
+    "application/vnd.apple2.dsk",
+    "application/vnd.apple2.bin",
+    "application/vnd.rom",
     ]
 
 pretty_mime = {
     "application/vnd.atari8bit.atr": "Atari 8-bit Disk Image",
     "application/vnd.atari8bit.xex": "Atari 8-bit Executable",
     "application/vnd.atari8bit.cart": "Atari 8-bit Cartridge",
-    "application/vnd.atari8bit.5200_cart":"Atari 5200 Cartridge",
+    "application/vnd.atari5200.cart": "Atari 5200 Cartridge",
+    "application/vnd.atari2600.cart": "Atari 2600 Cartridge",
+    "application/vnd.atari2600.starpath": "Atari 2600 Starpath Cassette",
+    "application/vnd.vectrex": "GCE Vectrex Cartridge",
     "application/vnd.mame_rom": "MAME",
+    "application/vnd.rom": "ROM Image",
     "application/vnd.apple2.dsk": "Apple ][ Disk Image",
     "application/vnd.apple2.bin": "Apple ][ Binary",
 }
@@ -292,20 +406,21 @@ for k in sizes:
     for c in grouped_carts[k]:
         t = c[0]
         name = c[1]
-        kclass = type("AtariCartSegmentParser%d" % t, (AtariCartSegmentParser,), {'cart_type': t, 'cart_info': c, 'menu_name': "%s Cartridge" % name})
         if "5200" in name:
-            key = "application/vnd.atari8bit.5200_cart"
+            key = "application/vnd.atari5200.cart"
+            kclass = type("Atari5200CartSegmentParser%d" % t, (Atari5200CartParser, AtariCartSegmentParser), {'cart_type': t, 'cart_info': c, 'menu_name': "%s Cartridge" % name})
         else:
             key = "application/vnd.atari8bit.cart"
+            kclass = type("Atari8bitCartSegmentParser%d" % t, (Atari8bitCartParser, AtariCartSegmentParser), {'cart_type': t, 'cart_info': c, 'menu_name': "%s Cartridge" % name})
         mime_parsers[key].append(kclass)
 
 
 known_segment_parsers = [DefaultSegmentParser]
-for mime in mime_parse_order:
+for mime in mime_display_order:
     known_segment_parsers.extend(mime_parsers[mime])
 
 
 def iter_known_segment_parsers():
     yield "application/octet-stream", "", [DefaultSegmentParser]
-    for mime in mime_parse_order:
+    for mime in mime_display_order:
         yield mime, pretty_mime[mime], mime_parsers[mime]

atrcopy/segments.py

@@ -76,7 +76,7 @@ class BSAVESaver:
         header = np.empty(2, dtype="<u2")
         header[0] = segment.origin
         header[1] = len(data)
-        print("binary data: %x bytes at %x" % (header[1], header[0]))
+        log.debug("binary data: %x bytes at %x" % (header[1], header[0]))
         return header.tobytes() + segment.tobytes()
 
 
@@ -414,7 +414,8 @@ class DefaultSegment:
 
     def set_raw(self, rawdata):
         if type(rawdata) != SegmentData:
-            log.warning(f"data not in SegmentData format {rawdata}, {type(rawdata)}")
+            log.warning(f"data not in SegmentData format: {type(rawdata)}")
+            rawdata = SegmentData(rawdata)
         self.rawdata = rawdata
         self.update_raw_pointers()
 
@@ -522,7 +523,7 @@ class DefaultSegment:
         """
         if hasattr(self, 'start_addr'):
             self.origin = self.start_addr
-            print(f"moving start_addr to origin: {self.start_addr}")
+            log.debug(f"moving start_addr to origin: {self.start_addr}")
             delattr(self, 'start_addr')
 
     def reconstruct_raw(self, rawdata):
@@ -552,7 +553,7 @@ class DefaultSegment:
             r = r.get_indexed[other.order]
         return r
 
-    def serialize_extra_to_dict(self, mdict):
+    def serialize_session(self, mdict):
         """Save extra metadata to a dict so that it can be serialized
 
         This is not saved by __getstate__ because child segments will point to
@@ -572,7 +573,7 @@ class DefaultSegment:
         # pairs
         mdict["comments"] = self.get_sorted_comments()
 
-    def restore_extra_from_dict(self, e):
+    def restore_session(self, e):
         if 'comments' in e:
             for k, v in e['comments']:
                 self.rawdata.extra.comments[k] = v

atrcopy/standard_delivery.py

@@ -165,7 +165,7 @@ from . fstbt import fstbt
 
 def get_fstbt_code(data, address_list, run_addr):
     pointer = len(fstbt)
-    data[0:pointer] = np.fromstring(fstbt, dtype=np.uint8)
+    data[0:pointer] = np.frombuffer(fstbt, dtype=np.uint8)
     hi, lo = divmod(run_addr, 256)
     data[pointer:pointer + 2] = (lo, hi)
     address_list.append(0xc0)  # last sector flag

atrcopy/templates/dos2dd.atr.inf

@@ -1 +1 @@
-{"task": "hex_edit", "description": "Atari 8-bit DOS 2 double density (180K), empty VTOC", "label": "Atari DOS 2 DD (180K) blank image", "ext": "atr"}
+{"type": "new file", "task": "hex_edit", "description": "Atari 8-bit DOS 2 double density (180K), empty VTOC", "label": "Atari DOS 2 DD (180K) blank image", "ext": "atr"}

atrcopy/templates/dos2ed+2.5.atr.inf

@@ -1 +1 @@
-{"task": "hex_edit", "description": "Atari 8-bit DOS 2 enhanced density (130K) DOS 2.5 system disk", "label": "Atari DOS 2.5 ED (130K) system disk", "ext": "atr"}
+{"type": "new file", "task": "hex_edit", "description": "Atari 8-bit DOS 2 enhanced density (130K) DOS 2.5 system disk", "label": "Atari DOS 2.5 ED (130K) system disk", "ext": "atr"}

atrcopy/templates/dos2ed.atr.inf

@@ -1 +1 @@
-{"task": "hex_edit", "description": "Atari 8-bit DOS 2 enhanced density (130K), empty VTOC", "label": "Atari DOS 2 ED (130K) blank image", "ext": "atr"}
+{"type": "new file", "task": "hex_edit", "description": "Atari 8-bit DOS 2 enhanced density (130K), empty VTOC", "label": "Atari DOS 2 ED (130K) blank image", "ext": "atr"}

atrcopy/templates/dos2sd+2.0s.atr.inf

@@ -1 +1 @@
-{"task": "hex_edit", "description": "Atari 8-bit DOS 2 single density (90K) DOS 2.0S system disk", "label": "Atari DOS 2.0S SD (90K) system disk", "ext": "atr"}
+{"type": "new file", "task": "hex_edit", "description": "Atari 8-bit DOS 2 single density (90K) DOS 2.0S system disk", "label": "Atari DOS 2.0S SD (90K) system disk", "ext": "atr"}

atrcopy/templates/dos2sd.atr.inf

@@ -1 +1 @@
-{"task": "hex_edit", "description": "Atari 8-bit DOS 2 single density (90K), empty VTOC", "label": "Atari DOS 2 SD (90K) blank image", "ext": "atr"}
+{"type": "new file", "task": "hex_edit", "description": "Atari 8-bit DOS 2 single density (90K), empty VTOC", "label": "Atari DOS 2 SD (90K) blank image", "ext": "atr"}

atrcopy/templates/dos33.dsk.inf

@@ -1 +1 @@
-{"task": "hex_edit", "description": "Apple ][ DOS 3.3 (140K) standard RWTS, empty VTOC", "label": "Apple DOS 3.3 (140K) blank image", "ext": "dsk"}
+{"type": "new file", "task": "hex_edit", "description": "Apple ][ DOS 3.3 (140K) standard RWTS, empty VTOC", "label": "Apple DOS 3.3 (140K) blank image", "ext": "dsk"}

atrcopy/templates/dos33autobrun.dsk.inf

@@ -1 +1 @@
-{"task": "hex_edit", "description": "Apple ][ DOS 3.3 (140K) disk image for binary program development: HELLO sets fullscreen HGR and calls BRUN on user-supplied AUTOBRUN binary file", "label": "Apple DOS 3.3 (140K) AUTOBRUN image", "ext": "dsk"}
+{"type": "new file", "task": "hex_edit", "description": "Apple ][ DOS 3.3 (140K) disk image for binary program development: HELLO sets fullscreen HGR and calls BRUN on user-supplied AUTOBRUN binary file", "label": "Apple DOS 3.3 (140K) AUTOBRUN image", "ext": "dsk"}

atrcopy/utils.py

@@ -29,7 +29,7 @@ def to_numpy(value):
     if type(value) is np.ndarray:
         return value
     elif type(value) is bytes:
-        return np.fromstring(value, dtype=np.uint8)
+        return np.copy(np.frombuffer(value, dtype=np.uint8))
     elif type(value) is list:
     	return np.asarray(value, dtype=np.uint8)
     raise TypeError("Can't convert to numpy data")

gen-sha.py

@@ -0,0 +1,47 @@
+#!/usr/bin/env python
+import os
+import sys
+import hashlib
+from collections import defaultdict
+import pprint
+
+def parse(filename, mime):
+    data = open(filename, 'rb').read()
+    h = hashlib.sha1(data).digest()
+    name = os.path.basename(os.path.splitext(filename)[0])
+    return len(data), h, name
+
+
+
+if __name__ == '__main__':
+    source = "atrcopy/signatures.py"
+    try:
+        with open(source, 'r') as fh:
+            source_text = fh.read()
+    except OSError:
+        source_text = "sha1_signatures = {}"
+    try:
+        exec(source_text)
+    except:
+        raise
+    print(sha1_signatures)
+    mime = sys.argv[1]
+    new_signatures = defaultdict(dict)
+    new_signatures.update(sha1_signatures)
+    for filename in sys.argv[2:]:
+        print(f"parsing {filename}")
+        size, hash_string, name = parse(filename, mime)
+        print(f"{size} {hash_string} {mime} {name}")
+        new_signatures[size][hash_string] = (mime, name)
+    lines = []
+    lines.append("sha1_signatures = {")
+    for k,v in sorted(new_signatures.items()):
+        lines.append(f"{k}: {{")
+        for h,n in sorted(v.items(), key=lambda a:(a[1], a[0])):
+            lines.append(f"  {h}: {n},")
+        lines.append("},")
+    lines.append("}  # end sha1_signatures\n")
+
+    print("\n".join(lines))
+    with open(source, 'w') as fh:
+        fh.write("\n".join(lines))

scripts/atrcopy

@@ -1,6 +1,12 @@
 #!/usr/bin/env python
 
 if __name__ == "__main__":
-    import atrcopy
+    import sys
+    if sys.version_info < (3, 6, 0):
+        print("atrcopy requires Python 3.6 or greater to run; this is Python %s" % ".".join([str(v) for v in sys.version_info[0:2]]))
+        if sys.version_info[0] == 2:
+            print("Python 2 support was dropped with atrcopy 7.0, so you can either use:\n\n    pip install \"atrcopy<7.0\"\n\nto install a version compatible with Python 2, or install Python 3.6 or higher.")
+    else:
+        import atrcopy
     
-    atrcopy.run()
+        atrcopy.run()

setup.py

@@ -5,7 +5,8 @@ try:
 except ImportError:
     from distutils.core import setup
 
-exec(open('atrcopy/_metadata.py').read())
+exec(compile(open('atrcopy/_version.py').read(), 'atrcopy/_version.py', 'exec'))
+exec(compile(open('atrcopy/_metadata.py').read(), 'atrcopy/_metadata.py', 'exec'))
 
 with open("README.rst", "r") as fp:
     long_description = fp.read()
@@ -23,16 +24,18 @@ setup(name="atrcopy",
         packages=["atrcopy"],
         include_package_data=True,
         scripts=scripts,
+        entry_points={"sawx.loaders": 'atrcopy = atrcopy.omnivore_loader'},
         description="Utility to manage file systems on Atari 8-bit (DOS 2) and Apple ][ (DOS 3.3) disk images.",
         long_description=long_description,
-        license="GPL",
+        license="MPL 2.0",
         classifiers=[
             "Programming Language :: Python :: 3.6",
             "Intended Audience :: Developers",
-            "License :: OSI Approved :: GNU General Public License (GPL)",
+            "License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)",
             "Topic :: Software Development :: Libraries",
             "Topic :: Utilities",
         ],
+        python_requires = '>=3.6',
         install_requires = [
             'numpy',
         ],

test/pytest.ini

@@ -1,2 +1,2 @@
-[pytest]
+#[pytest]
-addopts = --cov=atrcopy --cov-report html --cov-report term
+#addopts = --cov=atrcopy --cov-report html --cov-report term

test/test_add_file.py

@@ -25,7 +25,7 @@ class BaseFilesystemModifyTest:
             filename = "%s%d.BIN" % (prefix, count)
             self.image.write_file(filename, None, data)
             assert len(self.image.files) == orig_num_files + count
-            data2 = np.fromstring(self.image.find_file(filename), dtype=np.uint8)
+            data2 = np.frombuffer(self.image.find_file(filename), dtype=np.uint8)
             assert np.array_equal(data, data2[0:len(data)])
             count += 1
 
@@ -33,7 +33,7 @@ class BaseFilesystemModifyTest:
         count = 1
         for data in entries:
             filename = "%s%d.BIN" % (prefix, count)
-            data2 = np.fromstring(self.image.find_file(filename), dtype=np.uint8)
+            data2 = np.frombuffer(self.image.find_file(filename), dtype=np.uint8)
             assert np.array_equal(data, data2[0:len(data)])
             count += 1
             filenames.append(filename)
@@ -50,7 +50,7 @@ class BaseFilesystemModifyTest:
         self.image.write_file("TEST.XEX", None, data)
         assert len(self.image.files) == self.num_files_in_sample + 1
 
-        data2 = np.fromstring(self.image.find_file("TEST.XEX"), dtype=np.uint8)
+        data2 = np.frombuffer(self.image.find_file("TEST.XEX"), dtype=np.uint8)
         assert np.array_equal(data, data2[0:len(data)])
 
     def test_50k(self):

test/test_cart.py

@@ -3,8 +3,8 @@ from __future__ import division
 from builtins import object
 from mock import *
 
-from atrcopy import AtariCartImage, SegmentData
+from atrcopy import AtariCartImage, SegmentData, RomImage, errors
-from atrcopy import errors
+from atrcopy.cartridge import known_cart_types
 
 
 class TestAtariCart:
@@ -17,26 +17,24 @@ class TestAtariCart:
         data[4:8].view(">u4")[0] = cart_type
         return data
 
-    def test_unbanked(self):
+    @pytest.mark.parametrize("k_size,cart_type", [
-        carts = [
             (8, 1),
             (16, 2),
             (8, 21),
             (2, 57),
             (4, 58),
             (4, 59),
-        ]
+        ])
-        for k_size, cart_type in carts:
+    def test_unbanked(self, k_size, cart_type):
-            data = self.get_cart(k_size, cart_type)
+        data = self.get_cart(k_size, cart_type)
-            rawdata = SegmentData(data)
+        rawdata = SegmentData(data)
-            image = AtariCartImage(rawdata, cart_type)
+        image = AtariCartImage(rawdata, cart_type)
-            image.parse_segments()
+        image.parse_segments()
-            assert len(image.segments) == 2
+        assert len(image.segments) == 2
-            assert len(image.segments[0]) == 16
+        assert len(image.segments[0]) == 16
-            assert len(image.segments[1]) == k_size * 1024
+        assert len(image.segments[1]) == k_size * 1024
 
-    def test_banked(self):
+    @pytest.mark.parametrize("k_size,main_size,banked_size,cart_type", [
-        carts = [
             (32, 8, 8, 12),
             (64, 8, 8, 13),
             (64, 8, 8, 67),
@@ -44,16 +42,16 @@ class TestAtariCart:
             (256, 8, 8, 23),
             (512, 8, 8, 24),
             (1024, 8, 8, 25),
-        ]
+        ])
-        for k_size, main_size, banked_size, cart_type in carts:
+    def test_banked(self, k_size, main_size, banked_size, cart_type):
-            data = self.get_cart(k_size, cart_type)
+        data = self.get_cart(k_size, cart_type)
-            rawdata = SegmentData(data)
+        rawdata = SegmentData(data)
-            image = AtariCartImage(rawdata, cart_type)
+        image = AtariCartImage(rawdata, cart_type)
-            image.parse_segments()
+        image.parse_segments()
-            assert len(image.segments) == 1 + 1 + (k_size - main_size) //banked_size
+        assert len(image.segments) == 1 + 1 + (k_size - main_size) //banked_size
-            assert len(image.segments[0]) == 16
+        assert len(image.segments[0]) == 16
-            assert len(image.segments[1]) == main_size * 1024
+        assert len(image.segments[1]) == main_size * 1024
-            assert len(image.segments[2]) == banked_size * 1024
+        assert len(image.segments[2]) == banked_size * 1024
 
     def test_bad(self):
         k_size = 32
@@ -74,9 +72,55 @@ class TestAtariCart:
             image = AtariCartImage(rawdata, 1337)
 
 
+class TestRomCart:
+    def setup(self):
+        pass
+
+    def get_rom(self, k_size):
+        data = np.zeros((k_size * 1024), dtype=np.uint8)
+        return data
+
+    @pytest.mark.parametrize("k_size", [1, 2, 4, 8, 16, 32, 64])
+    def test_typical_rom_sizes(self, k_size):
+        data = self.get_rom(k_size)
+        rawdata = SegmentData(data)
+        rom_image = RomImage(rawdata)
+        rom_image.strict_check()
+        rom_image.parse_segments()
+        assert len(rom_image.segments) == 1
+        assert len(rom_image.segments[0]) == k_size * 1024
+
+    @pytest.mark.parametrize("k_size", [1, 2, 4, 8, 16, 32, 64])
+    def test_invalid_rom_sizes(self, k_size):
+        data = np.zeros((k_size * 1024) + 17, dtype=np.uint8)
+        rawdata = SegmentData(data)
+        with pytest.raises(errors.InvalidDiskImage):
+            rom_image = RomImage(rawdata)
+
+    @pytest.mark.parametrize("cart", known_cart_types)
+    def test_conversion_to_atari_cart(self, cart):
+        cart_type = cart[0]
+        name = cart[1]
+        k_size = cart[2]
+        if "Bounty" in name:
+            return
+        data = self.get_rom(k_size)
+        rawdata = SegmentData(data)
+        rom_image = RomImage(rawdata)
+        rom_image.strict_check()
+        rom_image.parse_segments()
+        new_cart_image = AtariCartImage(rawdata, cart_type)
+        new_cart_image.relaxed_check()
+        new_cart_image.parse_segments()
+        assert new_cart_image.header.valid
+        s = new_cart_image.create_emulator_boot_segment()
+        assert len(s) == len(rawdata) + new_cart_image.header.nominal_length
+        assert s[0:4].tobytes() == b'CART'
+        assert s[4:8].view(dtype=">u4") == cart_type
 
 
 if __name__ == "__main__":
+    from atrcopy.parsers import mime_parse_order
     print("\n".join(mime_parse_order))
 
     t = TestAtariCart()

test/test_create.py

@@ -41,9 +41,9 @@ class BaseCreateTest:
         try:
             _ = issubclass(errors.AtrError, expected)
             with pytest.raises(errors.InvalidBinaryFile) as e:
-                file_data, filetype = image.create_executable_file_image(segments, run_addr)
+                file_data, filetype = image.create_executable_file_image(sample_file, segments, run_addr)
         except TypeError:
-            file_data, filetype = image.create_executable_file_image(segments, run_addr)
+            file_data, filetype = image.create_executable_file_image(sample_file, segments, run_addr)
             print(image)
             print(file_data, filetype)
             assert len(file_data) == expected

test/test_serialize.py

@@ -47,15 +47,15 @@ class TestSegment:
         s.set_user_data([r], 4, 99)
 
         out = dict()
-        s.serialize_extra_to_dict(out)
+        s.serialize_session(out)
         print("saved", out)
 
         data = np.ones([4000], dtype=np.uint8)
         r = SegmentData(data)
         s2 = DefaultSegment(r, 0)
-        s2.restore_extra_from_dict(out)
+        s2.restore_session(out)
         out2 = dict()
-        s2.serialize_extra_to_dict(out2)
+        s2.serialize_session(out2)
         print("loaded", out2)
         assert out == out2