Initial checkin
This commit is contained in:
marty-emmanuel
commit
e216b0c544
|
|
@ -0,0 +1,10 @@
|
|||
bin
|
||||
vault
|
||||
packed_lzsa
|
||||
packed_lz4
|
||||
packed_lizard
|
||||
packed_lz5
|
||||
packed_zx7
|
||||
packed_rnc
|
||||
obj
|
||||
lzsa
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
## creative commons
|
||||
|
||||
# CC0 1.0 Universal
|
||||
|
||||
CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED HEREUNDER.
|
||||
|
||||
### Statement of Purpose
|
||||
|
||||
The laws of most jurisdictions throughout the world automatically confer exclusive Copyright and Related Rights (defined below) upon the creator and subsequent owner(s) (each and all, an "owner") of an original work of authorship and/or a database (each, a "Work").
|
||||
|
||||
Certain owners wish to permanently relinquish those rights to a Work for the purpose of contributing to a commons of creative, cultural and scientific works ("Commons") that the public can reliably and without fear of later claims of infringement build upon, modify, incorporate in other works, reuse and redistribute as freely as possible in any form whatsoever and for any purposes, including without limitation commercial purposes. These owners may contribute to the Commons to promote the ideal of a free culture and the further production of creative, cultural and scientific works, or to gain reputation or greater distribution for their Work in part through the use and efforts of others.
|
||||
|
||||
For these and/or other purposes and motivations, and without any expectation of additional consideration or compensation, the person associating CC0 with a Work (the "Affirmer"), to the extent that he or she is an owner of Copyright and Related Rights in the Work, voluntarily elects to apply CC0 to the Work and publicly distribute the Work under its terms, with knowledge of his or her Copyright and Related Rights in the Work and the meaning and intended legal effect of CC0 on those rights.
|
||||
|
||||
1. __Copyright and Related Rights.__ A Work made available under CC0 may be protected by copyright and related or neighboring rights ("Copyright and Related Rights"). Copyright and Related Rights include, but are not limited to, the following:
|
||||
|
||||
i. the right to reproduce, adapt, distribute, perform, display, communicate, and translate a Work;
|
||||
|
||||
ii. moral rights retained by the original author(s) and/or performer(s);
|
||||
|
||||
iii. publicity and privacy rights pertaining to a person's image or likeness depicted in a Work;
|
||||
|
||||
iv. rights protecting against unfair competition in regards to a Work, subject to the limitations in paragraph 4(a), below;
|
||||
|
||||
v. rights protecting the extraction, dissemination, use and reuse of data in a Work;
|
||||
|
||||
vi. database rights (such as those arising under Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, and under any national implementation thereof, including any amended or successor version of such directive); and
|
||||
|
||||
vii. other similar, equivalent or corresponding rights throughout the world based on applicable law or treaty, and any national implementations thereof.
|
||||
|
||||
2. __Waiver.__ To the greatest extent permitted by, but not in contravention of, applicable law, Affirmer hereby overtly, fully, permanently, irrevocably and unconditionally waives, abandons, and surrenders all of Affirmer's Copyright and Related Rights and associated claims and causes of action, whether now known or unknown (including existing as well as future claims and causes of action), in the Work (i) in all territories worldwide, (ii) for the maximum duration provided by applicable law or treaty (including future time extensions), (iii) in any current or future medium and for any number of copies, and (iv) for any purpose whatsoever, including without limitation commercial, advertising or promotional purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each member of the public at large and to the detriment of Affirmer's heirs and successors, fully intending that such Waiver shall not be subject to revocation, rescission, cancellation, termination, or any other legal or equitable action to disrupt the quiet enjoyment of the Work by the public as contemplated by Affirmer's express Statement of Purpose.
|
||||
|
||||
3. __Public License Fallback.__ Should any part of the Waiver for any reason be judged legally invalid or ineffective under applicable law, then the Waiver shall be preserved to the maximum extent permitted taking into account Affirmer's express Statement of Purpose. In addition, to the extent the Waiver is so judged Affirmer hereby grants to each affected person a royalty-free, non transferable, non sublicensable, non exclusive, irrevocable and unconditional license to exercise Affirmer's Copyright and Related Rights in the Work (i) in all territories worldwide, (ii) for the maximum duration provided by applicable law or treaty (including future time extensions), (iii) in any current or future medium and for any number of copies, and (iv) for any purpose whatsoever, including without limitation commercial, advertising or promotional purposes (the "License"). The License shall be deemed effective as of the date CC0 was applied by Affirmer to the Work. Should any part of the License for any reason be judged legally invalid or ineffective under applicable law, such partial invalidity or ineffectiveness shall not invalidate the remainder of the License, and in such case Affirmer hereby affirms that he or she will not (i) exercise any of his or her remaining Copyright and Related Rights in the Work or (ii) assert any associated claims and causes of action with respect to the Work, in either case contrary to Affirmer's express Statement of Purpose.
|
||||
|
||||
4. __Limitations and Disclaimers.__
|
||||
|
||||
a. No trademark or patent rights held by Affirmer are waived, abandoned, surrendered, licensed or otherwise affected by this document.
|
||||
|
||||
b. Affirmer offers the Work as-is and makes no representations or warranties of any kind concerning the Work, express, implied, statutory or otherwise, including without limitation warranties of title, merchantability, fitness for a particular purpose, non infringement, or the absence of latent or other defects, accuracy, or the present or absence of errors, whether or not discoverable, all to the greatest extent permissible under applicable law.
|
||||
|
||||
c. Affirmer disclaims responsibility for clearing rights of other persons that may apply to the Work or any use thereof, including without limitation any person's Copyright and Related Rights in the Work. Further, Affirmer disclaims responsibility for obtaining any necessary consents, permissions or other rights required for any use of the Work.
|
||||
|
||||
d. Affirmer understands and acknowledges that Creative Commons is not a party to this document and has no duty or obligation with respect to this CC0 or use of the Work.
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
Copyright (c) 2019 Emmanuel Marty
|
||||
|
||||
This software is provided 'as-is', without any express or implied warranty. In
|
||||
no event will the authors be held liable for any damages arising from the use of
|
||||
this software.
|
||||
|
||||
Permission is granted to anyone to use this software for any purpose, including
|
||||
commercial applications, and to alter it and redistribute it freely, subject to
|
||||
the following restrictions:
|
||||
|
||||
1. The origin of this software must not be misrepresented; you must not claim
|
||||
that you wrote the original software. If you use this software in a product,
|
||||
an acknowledgment in the product documentation would be appreciated but is
|
||||
not required.
|
||||
|
||||
2. Altered source versions must be plainly marked as such, and must not be
|
||||
misrepresented as being the original software.
|
||||
|
||||
3. This notice may not be removed or altered from any source distribution.
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
CC=clang
|
||||
CFLAGS=-O3 -fomit-frame-pointer -Isrc/libdivsufsort/include -Isrc -DHAVE_CONFIG_H
|
||||
OBJDIR=obj
|
||||
LDFLAGS=
|
||||
STRIP=strip
|
||||
|
||||
$(OBJDIR)/%.o: src/../%.c
|
||||
@mkdir -p '$(@D)'
|
||||
$(CC) $(CFLAGS) -c $< -o $@
|
||||
|
||||
APP := lzsa
|
||||
|
||||
OBJS := $(OBJDIR)/src/main.o
|
||||
OBJS += $(OBJDIR)/src/shrink.o
|
||||
OBJS += $(OBJDIR)/src/expand.o
|
||||
OBJS += $(OBJDIR)/src/libdivsufsort/lib/divsufsort.o
|
||||
OBJS += $(OBJDIR)/src/libdivsufsort/lib/sssort.o
|
||||
OBJS += $(OBJDIR)/src/libdivsufsort/lib/trsort.o
|
||||
OBJS += $(OBJDIR)/src/libdivsufsort/lib/utils.o
|
||||
|
||||
all: $(APP)
|
||||
|
||||
$(APP): $(OBJS)
|
||||
@mkdir -p ../../bin/posix
|
||||
$(CC) $^ $(LDFLAGS) -o $(APP)
|
||||
$(STRIP) $(APP)
|
||||
|
||||
clean:
|
||||
@rm -rf $(APP) $(OBJDIR)
|
||||
|
||||
|
|
@ -0,0 +1,121 @@
|
|||
LZSA is a byte-aligned compression format that is specifically engineered for very fast decompression on 8-bit systems. It can compress files of any size by using blocks of a maximum size of 64 Kb with block-interdependent compression and up to 64 Kb of back-references for matches.
|
||||
|
||||
The LZSA compression tool uses an aggressive optimal packing strategy to try to find the sequence of commands that gives the smallest packed file that decompresses to the original while maintaining the maximum possible decompression speed.
|
||||
|
||||
Compression ratio comparison between LZSA and other optimal packers, for a workload composed of ZX Spectrum and C64 files:
|
||||
|
||||
ZX7 57,36% (entropy coding)
|
||||
LZ5 1.4.1 59,82%
|
||||
LZSA 60,84% <------ (single byte stream)
|
||||
Lizard -29 64,14% (rep-match, 4 byte streams)
|
||||
LZ4_HC -19 -B4 -BD 64,5% (single byte stream)
|
||||
Uncompressed 100%
|
||||
|
||||
As an example of LZSA's simplicity, a size-optimized decompressor on 8088 has been implemented in 105 bytes.
|
||||
|
||||
The compressor is approximately 2X slower than LZ4_HC but compresses better while maintaining similar decompression speeds and decompressor simplicity.
|
||||
|
||||
The main differences with the LZ4 compression format are:
|
||||
|
||||
* The use of short (8-bit) match offsets where possible. The match-finder and optimizer cooperate to try and use the shortest match offsets possible.
|
||||
* Shorter encoding of lengths. As blocks are maximum 64 Kb in size, lengths can only be up to 64 Kb.
|
||||
* As a result of the smaller commands due to the possibly shorter match offsets, a minimum match size of 3 bytes instead of 4. The use of small matches is driven by the optimizer, and used where they provide gains.
|
||||
|
||||
Inspirations:
|
||||
|
||||
* [LZ4](https://github.com/lz4/lz4) by Yann Collet.
|
||||
* [LZ5/Lizard](https://github.com/inikep/lizard) by Przemyslaw Skibinski and Yann Collet.
|
||||
* The suffix array intervals in [Wimlib](https://wimlib.net/git/?p=wimlib;a=tree) by Eric Biggers.
|
||||
|
||||
License:
|
||||
|
||||
* The LZSA code is available under the Zlib license.
|
||||
* The compressor (shrink.c) is available under the CC0 license due to using portions of code from Eric Bigger's Wimlib in the suffix array-based matchfinder.
|
||||
|
||||
# Stream format
|
||||
|
||||
The stream format is composed of:
|
||||
* a header
|
||||
* one or more frames
|
||||
* a footer
|
||||
|
||||
# Header format
|
||||
|
||||
The header contains a signature and a traits byte:
|
||||
|
||||
0 1 2 3 4
|
||||
0x7b 0x9e 0x0f 0xd7 0x00
|
||||
<--- signature ---> <- traits ->
|
||||
|
||||
The traits are set to 0x00 for this version of the format.
|
||||
|
||||
# Frame format
|
||||
|
||||
Each frame contains a 3-byte length followed by block data that expands to up to 64 Kb of decompressed data.
|
||||
|
||||
0 1 2
|
||||
DSZ0 DSZ1 U|DSZ2
|
||||
|
||||
* DSZ0 (length byte 0) contains bits 0-7 of the block data size
|
||||
* DSZ1 (length byte 1) contains bits 8-15 of the block data size
|
||||
* DSZ2 (bit 0 of length byte 2) contains bit 16 of the block data size
|
||||
* U (bit 7 of length byte 2) is set if the block data is uncompressed, and clear if the block data is compressed.
|
||||
* Bits 1..6 of length byte 2 are currently undefined and must be set to 0.
|
||||
|
||||
# Block data format
|
||||
|
||||
LZSA blocks are composed from consecutive commands. Each command follows this format:
|
||||
|
||||
* <token: O|LLL|MMMM>
|
||||
* <optional extra literal length>
|
||||
* <literal values>
|
||||
* <match offset low>
|
||||
* <optional match offset high>
|
||||
* <optional extra match length>
|
||||
|
||||
**token**
|
||||
|
||||
The token byte is broken down into three parts:
|
||||
|
||||
7 6 5 4 3 2 1 0
|
||||
O L L L M M M M
|
||||
|
||||
* O: set for a 2-byte match offset, clear for a 1-byte match offset
|
||||
* L: 3-bit literals length (0-6, or 7 if extended). If the number of literals for this command is 0 to 6, the length is encoded in the token and no extra bytes are required. Otherwise, a value of 7 is encoded and extra bytes follow as 'optional extra literal length'
|
||||
* M: 4-bit match length (0-14, or 15 if extended). Likewise, if the match length for this command is 0 to 14, it is directly encoded, otherwise 15 is stored and extra bytes follow as 'optional extra match length'.
|
||||
|
||||
**optional extra literal length**
|
||||
|
||||
If the literals length is 7 or more, the 'L' bits in the token form the value 7, and an extra byte follows here, with three possible types of value:
|
||||
|
||||
* 0-254: the value is added to the 7 stored in the token, to compose the final literals length. For instance a length of 206 will be stored as 7 in the token + a single byte with the value of 199, as 7 + 199 = 206.
|
||||
* 254: a second byte follows. The final literals value is 7 + 254 + the second byte. For instance, a literals length of 499 is encoded as 7 in the token, a byte with the value of 254, and a final byte with teh value of 238, as 7 + 254 + 238 = 499.
|
||||
* 255: a second and third byte follow, forming a little-endian 16-bit value. The final literals value is 7 + 255 + that 16-bit value. For instance, a literals length of 1024 is stored as 7 in the token, then byte values of 255, 250 and 2, as 7 + 255 + 250 + (2 * 256) = 1024.
|
||||
|
||||
**literal values**
|
||||
|
||||
Literal bytes, whose number is specified by the literals length, follow here. There can be zero literals in a command.
|
||||
|
||||
**match offset low**
|
||||
|
||||
The low 8 bits of the match offset follows.
|
||||
|
||||
**optional match offset high**
|
||||
|
||||
If the 'O' bit (bit 7) is set in the token, the high 8 bits of the match offset follow, otherwise they are understood to be all set to 0.
|
||||
|
||||
**important note regarding match offsets: off by 1**
|
||||
|
||||
Note that the match offset is *off by 1*: a value of 0 refers to the byte preceding the current output index (N-1). A value of 1 refers to tow bytes before the current output index (N-2) and so on. This is so that match offsets up to 256 can be encoded as a single byte, for extra compression.
|
||||
|
||||
**optional extra match length**
|
||||
|
||||
If the match length is 15 or more, the 'M' bits in the token form the value 15, and an extra byte follows here, with three possible types of value.
|
||||
|
||||
* 0-254: the value is added to the 15 stored in the token.
|
||||
* 254: a second byte follows. The final match length is 15 + 254 + the second byte.
|
||||
* 255: a second and third byte follow, forming a little-endian 16-bit value. The final match length is 15 + 255 + that 16-bit value.
|
||||
|
||||
# Footer format
|
||||
|
||||
The stream ends with an empty frame: the 3 length bytes are set to 0 and no block data follows.
|
||||
|
|
@ -0,0 +1,125 @@
|
|||
; decompress_small.S - space-efficient decompressor implementation for 8088
|
||||
;
|
||||
; Copyright (C) 2019 Emmanuel Marty
|
||||
;
|
||||
; This software is provided 'as-is', without any express or implied
|
||||
; warranty. In no event will the authors be held liable for any damages
|
||||
; arising from the use of this software.
|
||||
;
|
||||
; Permission is granted to anyone to use this software for any purpose,
|
||||
; including commercial applications, and to alter it and redistribute it
|
||||
; freely, subject to the following restrictions:
|
||||
;
|
||||
; 1. The origin of this software must not be misrepresented; you must not
|
||||
; claim that you wrote the original software. If you use this software
|
||||
; in a product, an acknowledgment in the product documentation would be
|
||||
; appreciated but is not required.
|
||||
; 2. Altered source versions must be plainly marked as such, and must not be
|
||||
; misrepresented as being the original software.
|
||||
; 3. This notice may not be removed or altered from any source distribution.
|
||||
|
||||
segment .text
|
||||
bits 16
|
||||
|
||||
; ---------------------------------------------------------------------------
|
||||
; Decompress LZSA frame
|
||||
; inputs:
|
||||
; * ds:si: LZSA frame
|
||||
; * es:di: output buffer
|
||||
; output:
|
||||
; * ax: decompressed size
|
||||
; ---------------------------------------------------------------------------
|
||||
|
||||
lzsa_decompress:
|
||||
push di ; remember decompression offset
|
||||
cld ; make string operations (lods, movs, stos..) move forward
|
||||
lodsw ; grab first 2 bytes of block size
|
||||
mov dx,ax ; keep block size in dx
|
||||
lodsb ; grab last byte of block size / compression flag
|
||||
test al,al ; if the compressed data is larger than 65,535 bytes, or if it is uncompressed, bail
|
||||
jne .done_decompressing
|
||||
add dx,si ; point at the end of the compressed data
|
||||
mov bp,dx ; remember it
|
||||
|
||||
xor cx,cx
|
||||
|
||||
.decode_token:
|
||||
mov ax,cx ; clear ah - cx is zero from above or from after rep movsw in .copy_match
|
||||
lodsb ; read token byte: O|LLL|MMMM
|
||||
mov dx,ax ; keep token in dl
|
||||
|
||||
and al,070H ; isolate literals length in token (LLL)
|
||||
mov cl,4
|
||||
shr al,cl ; shift match length into place
|
||||
|
||||
mov cx,ax ; copy literals length into cx
|
||||
cmp al,07H ; LITERALS_RUN_LEN?
|
||||
jne .copy_literals ; no, we have the full literals count from the token, go copy
|
||||
|
||||
call .get_varlen ; get complete literals length
|
||||
|
||||
.copy_literals:
|
||||
rep movsb ; copy cx literals from ds:si to es:di
|
||||
|
||||
cmp si,bp ; did we reach the end of the compressed data?
|
||||
je .done_decompressing ; we did, bail. the last token does not include a match copy
|
||||
|
||||
test dl,dl ; check match offset size in token (O bit, ie. bit 7, the sign bit)
|
||||
js .get_long_offset
|
||||
|
||||
xor ah,ah ; Get 1-byte match offset
|
||||
lodsb
|
||||
jmp short .get_match_length
|
||||
|
||||
.get_long_offset:
|
||||
lodsw ; Get 2-byte match offset
|
||||
|
||||
.get_match_length:
|
||||
inc ax ; the match offset is stored off-by-1, increase it
|
||||
xchg ax,dx ; dx: match offset ax: original token
|
||||
and al,0FH ; isolate match length in token (MMMM)
|
||||
|
||||
mov cx,ax ; copy match length into cx
|
||||
cmp al,0FH ; MATCH_RUN_LEN?
|
||||
jne .copy_match ; no, we have the full match length from the token, go copy
|
||||
|
||||
call .get_varlen ; get complete match length
|
||||
|
||||
.copy_match:
|
||||
add cx,3 ; add MIN_MATCH_SIZE to get the final match length to copy
|
||||
push ds ; save ds:si (current pointer to compressed data)
|
||||
xchg si,ax
|
||||
push es
|
||||
pop ds
|
||||
mov si,di ; ds:si now points at back reference in output data
|
||||
sub si,dx
|
||||
rep movsb ; copy match
|
||||
xchg si,ax ; restore ds:si
|
||||
pop ds
|
||||
jmp short .decode_token ; go decode another token
|
||||
|
||||
.done_decompressing:
|
||||
pop ax ; retrieve the original decompression offset
|
||||
xchg ax,di ; compute decompressed size
|
||||
sub ax,di
|
||||
ret ; done
|
||||
|
||||
.get_varlen:
|
||||
lodsb ; grab extra length byte
|
||||
add cx,ax ; add extra length byte to length from token
|
||||
|
||||
cmp al,0FFH ; 3-byte extra length?
|
||||
je .large_varlen ; yes, go grab it
|
||||
|
||||
cmp al,0FEH ; 2-byte extra length?
|
||||
jne .varlen_done ; no, we have the full length now, bail
|
||||
|
||||
lodsb ; grab extra length byte
|
||||
jmp short .add_and_varlen_done ; go add it and bail
|
||||
|
||||
.large_varlen:
|
||||
lodsw ; grab 16-bit extra length
|
||||
.add_and_varlen_done:
|
||||
add cx,ax ; add to length from token
|
||||
.varlen_done:
|
||||
ret
|
||||
|
|
@ -0,0 +1,230 @@
|
|||
/*
|
||||
* expand.c - block decompressor implementation
|
||||
*
|
||||
* Copyright (C) 2019 Emmanuel Marty
|
||||
*
|
||||
* This software is provided 'as-is', without any express or implied
|
||||
* warranty. In no event will the authors be held liable for any damages
|
||||
* arising from the use of this software.
|
||||
*
|
||||
* Permission is granted to anyone to use this software for any purpose,
|
||||
* including commercial applications, and to alter it and redistribute it
|
||||
* freely, subject to the following restrictions:
|
||||
*
|
||||
* 1. The origin of this software must not be misrepresented; you must not
|
||||
* claim that you wrote the original software. If you use this software
|
||||
* in a product, an acknowledgment in the product documentation would be
|
||||
* appreciated but is not required.
|
||||
* 2. Altered source versions must be plainly marked as such, and must not be
|
||||
* misrepresented as being the original software.
|
||||
* 3. This notice may not be removed or altered from any source distribution.
|
||||
*/
|
||||
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include "format.h"
|
||||
#include "expand.h"
|
||||
|
||||
#ifdef _MSC_VER
|
||||
#define FORCE_INLINE __forceinline
|
||||
#else /* _MSC_VER */
|
||||
#define FORCE_INLINE __attribute__((always_inline))
|
||||
#endif /* _MSC_VER */
|
||||
|
||||
static inline FORCE_INLINE int lzsa_expand_literals_slow(const unsigned char **ppInBlock, const unsigned char *pInBlockEnd, int nLiterals, unsigned char **ppCurOutData, const unsigned char *pOutDataEnd) {
|
||||
const unsigned char *pInBlock = *ppInBlock;
|
||||
unsigned char *pCurOutData = *ppCurOutData;
|
||||
|
||||
if (nLiterals == LITERALS_RUN_LEN) {
|
||||
unsigned char nByte;
|
||||
|
||||
if (pInBlock >= pInBlockEnd) return -1;
|
||||
nByte = *pInBlock++;
|
||||
nLiterals += (int)((unsigned int)nByte);
|
||||
|
||||
if (nByte == 254) {
|
||||
if (pInBlock >= pInBlockEnd) return -1;
|
||||
nLiterals += (int)((unsigned int)*pInBlock++);
|
||||
}
|
||||
else if (nByte == 255) {
|
||||
int nLargeLiterals;
|
||||
|
||||
if ((pInBlock + 1) >= pInBlockEnd) return -1;
|
||||
nLargeLiterals = ((unsigned int)*pInBlock++);
|
||||
nLargeLiterals |= (((unsigned int)*pInBlock++) << 8);
|
||||
nLiterals += nLargeLiterals;
|
||||
}
|
||||
}
|
||||
|
||||
if (nLiterals != 0) {
|
||||
if ((pInBlock + nLiterals) > pInBlockEnd ||
|
||||
(pCurOutData + nLiterals) > pOutDataEnd) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
memcpy(pCurOutData, pInBlock, nLiterals);
|
||||
pInBlock += nLiterals;
|
||||
pCurOutData += nLiterals;
|
||||
}
|
||||
|
||||
*ppInBlock = pInBlock;
|
||||
*ppCurOutData = pCurOutData;
|
||||
return 0;
|
||||
}
|
||||
|
||||
static inline FORCE_INLINE int lzsa_expand_match_slow(const unsigned char **ppInBlock, const unsigned char *pInBlockEnd, const unsigned char *pSrc, int nMatchLen, unsigned char **ppCurOutData, const unsigned char *pOutDataEnd, const unsigned char *pOutDataFastEnd) {
|
||||
const unsigned char *pInBlock = *ppInBlock;
|
||||
unsigned char *pCurOutData = *ppCurOutData;
|
||||
|
||||
if (nMatchLen == MATCH_RUN_LEN) {
|
||||
unsigned char nByte;
|
||||
|
||||
if (pInBlock >= pInBlockEnd) return -1;
|
||||
nByte = *pInBlock++;
|
||||
nMatchLen += (int)((unsigned int)nByte);
|
||||
|
||||
if (nByte == 254) {
|
||||
if (pInBlock >= pInBlockEnd) return -1;
|
||||
nMatchLen += (int)((unsigned int)*pInBlock++);
|
||||
}
|
||||
else if (nByte == 255) {
|
||||
int nLargeMatchLen;
|
||||
|
||||
if ((pInBlock + 1) >= pInBlockEnd) return -1;
|
||||
nLargeMatchLen = ((unsigned int)*pInBlock++);
|
||||
nLargeMatchLen |= (((unsigned int)*pInBlock++) << 8);
|
||||
nMatchLen += nLargeMatchLen;
|
||||
}
|
||||
}
|
||||
|
||||
nMatchLen += MIN_MATCH_SIZE;
|
||||
|
||||
if ((pCurOutData + nMatchLen) > pOutDataEnd) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
if ((pSrc + 1) == pCurOutData && nMatchLen >= 16) {
|
||||
/* One-byte RLE */
|
||||
memset(pCurOutData, *pSrc, nMatchLen);
|
||||
pCurOutData += nMatchLen;
|
||||
}
|
||||
else {
|
||||
/* Do a deterministic, left to right byte copy instead of memcpy() so as to handle overlaps */
|
||||
|
||||
int nMaxFast = nMatchLen;
|
||||
if (nMaxFast > (pCurOutData - pSrc))
|
||||
nMaxFast = (int)(pCurOutData - pSrc);
|
||||
if ((pCurOutData + nMaxFast) > pOutDataFastEnd)
|
||||
nMaxFast = (int)(pOutDataFastEnd - pCurOutData);
|
||||
|
||||
if (nMaxFast > 0) {
|
||||
const unsigned char *pCopySrc = pSrc;
|
||||
unsigned char *pCopyDst = pCurOutData;
|
||||
const unsigned char *pCopyEndDst = pCurOutData + nMaxFast;
|
||||
|
||||
do {
|
||||
memcpy(pCopyDst, pCopySrc, 16);
|
||||
pCopySrc += 16;
|
||||
pCopyDst += 16;
|
||||
} while (pCopyDst < pCopyEndDst);
|
||||
|
||||
pCurOutData += nMaxFast;
|
||||
pSrc += nMaxFast;
|
||||
nMatchLen -= nMaxFast;
|
||||
}
|
||||
|
||||
while (nMatchLen >= 4) {
|
||||
*pCurOutData++ = *pSrc++;
|
||||
*pCurOutData++ = *pSrc++;
|
||||
*pCurOutData++ = *pSrc++;
|
||||
*pCurOutData++ = *pSrc++;
|
||||
nMatchLen -= 4;
|
||||
}
|
||||
while (nMatchLen > 0) {
|
||||
*pCurOutData++ = *pSrc++;
|
||||
nMatchLen--;
|
||||
}
|
||||
}
|
||||
|
||||
*ppInBlock = pInBlock;
|
||||
*ppCurOutData = pCurOutData;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int lzsa_expand_block(const unsigned char *pInBlock, int nBlockSize, unsigned char *pOutData, int nOutDataOffset, int nBlockMaxSize) {
|
||||
const unsigned char *pInBlockEnd = pInBlock + nBlockSize;
|
||||
const unsigned char *pInBlockFastEnd = pInBlock + nBlockSize - 16;
|
||||
unsigned char *pCurOutData = pOutData + nOutDataOffset;
|
||||
const unsigned char *pOutDataEnd = pCurOutData + nBlockMaxSize;
|
||||
const unsigned char *pOutDataFastEnd = pOutDataEnd - 16;
|
||||
|
||||
/* Fast loop */
|
||||
|
||||
while (pInBlock < pInBlockFastEnd && pCurOutData < pOutDataFastEnd) {
|
||||
const unsigned char token = *pInBlock++;
|
||||
int nLiterals = (int)((unsigned int)((token & 0x70) >> 4));
|
||||
|
||||
if (nLiterals < LITERALS_RUN_LEN) {
|
||||
memcpy(pCurOutData, pInBlock, 8);
|
||||
pInBlock += nLiterals;
|
||||
pCurOutData += nLiterals;
|
||||
}
|
||||
else {
|
||||
if (lzsa_expand_literals_slow(&pInBlock, pInBlockEnd, nLiterals, &pCurOutData, pOutDataEnd))
|
||||
return -1;
|
||||
}
|
||||
|
||||
if ((pInBlock + 2) <= pInBlockEnd) { /* The last token in the block does not include match information */
|
||||
int nMatchOffset;
|
||||
|
||||
nMatchOffset = ((unsigned int)*pInBlock++);
|
||||
if (token & 0x80)
|
||||
nMatchOffset |= (((unsigned int)*pInBlock++) << 8);
|
||||
nMatchOffset++;
|
||||
|
||||
const unsigned char *pSrc = pCurOutData - nMatchOffset;
|
||||
if (pSrc < pOutData)
|
||||
return -1;
|
||||
|
||||
int nMatchLen = (int)((unsigned int)(token & 0x0f));
|
||||
if (nMatchLen < (16 - MIN_MATCH_SIZE + 1) && (pSrc + MIN_MATCH_SIZE + nMatchLen) < pCurOutData) {
|
||||
memcpy(pCurOutData, pSrc, 16);
|
||||
pCurOutData += (MIN_MATCH_SIZE + nMatchLen);
|
||||
}
|
||||
else {
|
||||
if (lzsa_expand_match_slow(&pInBlock, pInBlockEnd, pSrc, nMatchLen, &pCurOutData, pOutDataEnd, pOutDataFastEnd))
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Slow loop for the remainder of the buffer */
|
||||
|
||||
while (pInBlock < pInBlockEnd) {
|
||||
const unsigned char token = *pInBlock++;
|
||||
int nLiterals = (int)((unsigned int)((token & 0x70) >> 4));
|
||||
|
||||
if (lzsa_expand_literals_slow(&pInBlock, pInBlockEnd, nLiterals, &pCurOutData, pOutDataEnd))
|
||||
return -1;
|
||||
|
||||
if ((pInBlock + 2) <= pInBlockEnd) { /* The last token in the block does not include match information */
|
||||
int nMatchOffset;
|
||||
|
||||
nMatchOffset = ((unsigned int)*pInBlock++);
|
||||
if (token & 0x80)
|
||||
nMatchOffset |= (((unsigned int)*pInBlock++) << 8);
|
||||
nMatchOffset++;
|
||||
|
||||
const unsigned char *pSrc = pCurOutData - nMatchOffset;
|
||||
if (pSrc < pOutData)
|
||||
return -1;
|
||||
|
||||
int nMatchLen = (int)((unsigned int)(token & 0x0f));
|
||||
if (lzsa_expand_match_slow(&pInBlock, pInBlockEnd, pSrc, nMatchLen, &pCurOutData, pOutDataEnd, pOutDataFastEnd))
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
return (int)(pCurOutData - (pOutData + nOutDataOffset));
|
||||
}
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
/*
|
||||
* expand.h - block decompressor definitions
|
||||
*
|
||||
* Copyright (C) 2019 Emmanuel Marty
|
||||
*
|
||||
* This software is provided 'as-is', without any express or implied
|
||||
* warranty. In no event will the authors be held liable for any damages
|
||||
* arising from the use of this software.
|
||||
*
|
||||
* Permission is granted to anyone to use this software for any purpose,
|
||||
* including commercial applications, and to alter it and redistribute it
|
||||
* freely, subject to the following restrictions:
|
||||
*
|
||||
* 1. The origin of this software must not be misrepresented; you must not
|
||||
* claim that you wrote the original software. If you use this software
|
||||
* in a product, an acknowledgment in the product documentation would be
|
||||
* appreciated but is not required.
|
||||
* 2. Altered source versions must be plainly marked as such, and must not be
|
||||
* misrepresented as being the original software.
|
||||
* 3. This notice may not be removed or altered from any source distribution.
|
||||
*/
|
||||
|
||||
#ifndef _EXPAND_H
|
||||
#define _EXPAND_H
|
||||
|
||||
int lzsa_expand_block(const unsigned char *pInBlock, int nBlockSize, unsigned char *pOutData, int nOutDataOffset, int nBlockMaxSize);
|
||||
|
||||
#endif /* _EXPAND_H */
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
/*
|
||||
* format.h - byte stream format definitions
|
||||
*
|
||||
* Copyright (C) 2019 Emmanuel Marty
|
||||
*
|
||||
* This software is provided 'as-is', without any express or implied
|
||||
* warranty. In no event will the authors be held liable for any damages
|
||||
* arising from the use of this software.
|
||||
*
|
||||
* Permission is granted to anyone to use this software for any purpose,
|
||||
* including commercial applications, and to alter it and redistribute it
|
||||
* freely, subject to the following restrictions:
|
||||
*
|
||||
* 1. The origin of this software must not be misrepresented; you must not
|
||||
* claim that you wrote the original software. If you use this software
|
||||
* in a product, an acknowledgment in the product documentation would be
|
||||
* appreciated but is not required.
|
||||
* 2. Altered source versions must be plainly marked as such, and must not be
|
||||
* misrepresented as being the original software.
|
||||
* 3. This notice may not be removed or altered from any source distribution.
|
||||
*/
|
||||
|
||||
#ifndef _FORMAT_H
|
||||
#define _FORMAT_H
|
||||
|
||||
#define MIN_MATCH_SIZE 3
|
||||
#define MIN_OFFSET 1
|
||||
#define MAX_OFFSET 0xffff
|
||||
#define LITERALS_RUN_LEN 7
|
||||
#define MATCH_RUN_LEN 15
|
||||
|
||||
#endif /* _FORMAT_H */
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
# Object files
|
||||
*.o
|
||||
*.ko
|
||||
*.obj
|
||||
*.elf
|
||||
|
||||
# Precompiled Headers
|
||||
*.gch
|
||||
*.pch
|
||||
|
||||
# Libraries
|
||||
*.lib
|
||||
*.a
|
||||
*.la
|
||||
*.lo
|
||||
|
||||
# Shared objects (inc. Windows DLLs)
|
||||
*.dll
|
||||
*.so
|
||||
*.so.*
|
||||
*.dylib
|
||||
|
||||
# Executables
|
||||
*.exe
|
||||
*.out
|
||||
*.app
|
||||
*.i*86
|
||||
*.x86_64
|
||||
*.hex
|
||||
|
||||
# CMake files/directories
|
||||
build/
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
# libdivsufsort Change Log
|
||||
|
||||
See full changelog at: https://github.com/y-256/libdivsufsort/commits
|
||||
|
||||
## [2.0.1] - 2010-11-11
|
||||
### Fixed
|
||||
* Wrong variable used in `divbwt` function
|
||||
* Enclose some string variables with double quotation marks in include/CMakeLists.txt
|
||||
* Fix typo in include/CMakeLists.txt
|
||||
|
||||
## 2.0.0 - 2008-08-23
|
||||
### Changed
|
||||
* Switch the build system to [CMake](http://www.cmake.org/)
|
||||
* Improve the performance of the suffix-sorting algorithm
|
||||
|
||||
### Added
|
||||
* OpenMP support
|
||||
* 64-bit version of divsufsort
|
||||
|
||||
[Unreleased]: https://github.com/y-256/libdivsufsort/compare/2.0.1...HEAD
|
||||
[2.0.1]: https://github.com/y-256/libdivsufsort/compare/2.0.0...2.0.1
|
||||
|
|
@ -0,0 +1,99 @@
|
|||
### cmake file for building libdivsufsort Package ###
|
||||
cmake_minimum_required(VERSION 2.4.4)
|
||||
set(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/CMakeModules")
|
||||
include(AppendCompilerFlags)
|
||||
|
||||
## Project information ##
|
||||
project(libdivsufsort C)
|
||||
set(PROJECT_VENDOR "Yuta Mori")
|
||||
set(PROJECT_CONTACT "yuta.256@gmail.com")
|
||||
set(PROJECT_URL "https://github.com/y-256/libdivsufsort")
|
||||
set(PROJECT_DESCRIPTION "A lightweight suffix sorting library")
|
||||
include(VERSION.cmake)
|
||||
|
||||
## CPack configuration ##
|
||||
set(CPACK_GENERATOR "TGZ;TBZ2;ZIP")
|
||||
set(CPACK_SOURCE_GENERATOR "TGZ;TBZ2;ZIP")
|
||||
include(ProjectCPack)
|
||||
|
||||
## Project options ##
|
||||
option(BUILD_SHARED_LIBS "Set to OFF to build static libraries" ON)
|
||||
option(BUILD_EXAMPLES "Build examples" ON)
|
||||
option(BUILD_DIVSUFSORT64 "Build libdivsufsort64" OFF)
|
||||
option(USE_OPENMP "Use OpenMP for parallelization" OFF)
|
||||
option(WITH_LFS "Enable Large File Support" ON)
|
||||
|
||||
## Installation directories ##
|
||||
set(LIB_SUFFIX "" CACHE STRING "Define suffix of directory name (32 or 64)")
|
||||
|
||||
set(CMAKE_INSTALL_RUNTIMEDIR "" CACHE PATH "Specify the output directory for dll runtimes (default is bin)")
|
||||
if(NOT CMAKE_INSTALL_RUNTIMEDIR)
|
||||
set(CMAKE_INSTALL_RUNTIMEDIR "${CMAKE_INSTALL_PREFIX}/bin")
|
||||
endif(NOT CMAKE_INSTALL_RUNTIMEDIR)
|
||||
|
||||
set(CMAKE_INSTALL_LIBDIR "" CACHE PATH "Specify the output directory for libraries (default is lib)")
|
||||
if(NOT CMAKE_INSTALL_LIBDIR)
|
||||
set(CMAKE_INSTALL_LIBDIR "${CMAKE_INSTALL_PREFIX}/lib${LIB_SUFFIX}")
|
||||
endif(NOT CMAKE_INSTALL_LIBDIR)
|
||||
|
||||
set(CMAKE_INSTALL_INCLUDEDIR "" CACHE PATH "Specify the output directory for header files (default is include)")
|
||||
if(NOT CMAKE_INSTALL_INCLUDEDIR)
|
||||
set(CMAKE_INSTALL_INCLUDEDIR "${CMAKE_INSTALL_PREFIX}/include")
|
||||
endif(NOT CMAKE_INSTALL_INCLUDEDIR)
|
||||
|
||||
set(CMAKE_INSTALL_PKGCONFIGDIR "" CACHE PATH "Specify the output directory for pkgconfig files (default is lib/pkgconfig)")
|
||||
if(NOT CMAKE_INSTALL_PKGCONFIGDIR)
|
||||
set(CMAKE_INSTALL_PKGCONFIGDIR "${CMAKE_INSTALL_LIBDIR}/pkgconfig")
|
||||
endif(NOT CMAKE_INSTALL_PKGCONFIGDIR)
|
||||
|
||||
## Build type ##
|
||||
if(NOT CMAKE_BUILD_TYPE)
|
||||
set(CMAKE_BUILD_TYPE "Release")
|
||||
elseif(CMAKE_BUILD_TYPE STREQUAL "Debug")
|
||||
set(CMAKE_VERBOSE_MAKEFILE ON)
|
||||
endif(NOT CMAKE_BUILD_TYPE)
|
||||
|
||||
## Compiler options ##
|
||||
if(MSVC)
|
||||
append_c_compiler_flags("/W4" "VC" CMAKE_C_FLAGS)
|
||||
append_c_compiler_flags("/Oi;/Ot;/Ox;/Oy" "VC" CMAKE_C_FLAGS_RELEASE)
|
||||
if(USE_OPENMP)
|
||||
append_c_compiler_flags("/openmp" "VC" CMAKE_C_FLAGS)
|
||||
endif(USE_OPENMP)
|
||||
elseif(BORLAND)
|
||||
append_c_compiler_flags("-w" "BCC" CMAKE_C_FLAGS)
|
||||
append_c_compiler_flags("-Oi;-Og;-Os;-Ov;-Ox" "BCC" CMAKE_C_FLAGS_RELEASE)
|
||||
else(MSVC)
|
||||
if(CMAKE_COMPILER_IS_GNUCC)
|
||||
append_c_compiler_flags("-Wall" "GCC" CMAKE_C_FLAGS)
|
||||
append_c_compiler_flags("-fomit-frame-pointer" "GCC" CMAKE_C_FLAGS_RELEASE)
|
||||
if(USE_OPENMP)
|
||||
append_c_compiler_flags("-fopenmp" "GCC" CMAKE_C_FLAGS)
|
||||
endif(USE_OPENMP)
|
||||
else(CMAKE_COMPILER_IS_GNUCC)
|
||||
append_c_compiler_flags("-Wall" "UNKNOWN" CMAKE_C_FLAGS)
|
||||
append_c_compiler_flags("-fomit-frame-pointer" "UNKNOWN" CMAKE_C_FLAGS_RELEASE)
|
||||
if(USE_OPENMP)
|
||||
append_c_compiler_flags("-fopenmp;-openmp;-omp" "UNKNOWN" CMAKE_C_FLAGS)
|
||||
endif(USE_OPENMP)
|
||||
endif(CMAKE_COMPILER_IS_GNUCC)
|
||||
endif(MSVC)
|
||||
|
||||
## Add definitions ##
|
||||
add_definitions(-DHAVE_CONFIG_H=1 -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS)
|
||||
|
||||
## Add subdirectories ##
|
||||
add_subdirectory(pkgconfig)
|
||||
add_subdirectory(include)
|
||||
add_subdirectory(lib)
|
||||
if(BUILD_EXAMPLES)
|
||||
add_subdirectory(examples)
|
||||
endif(BUILD_EXAMPLES)
|
||||
|
||||
## Add 'uninstall' target ##
|
||||
CONFIGURE_FILE(
|
||||
"${CMAKE_CURRENT_SOURCE_DIR}/CMakeModules/cmake_uninstall.cmake.in"
|
||||
"${CMAKE_CURRENT_BINARY_DIR}/CMakeModules/cmake_uninstall.cmake"
|
||||
IMMEDIATE @ONLY)
|
||||
ADD_CUSTOM_TARGET(uninstall
|
||||
"${CMAKE_COMMAND}" -P "${CMAKE_CURRENT_BINARY_DIR}/CMakeModules/cmake_uninstall.cmake")
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
include(CheckCSourceCompiles)
|
||||
include(CheckCXXSourceCompiles)
|
||||
|
||||
macro(append_c_compiler_flags _flags _name _result)
|
||||
set(SAFE_CMAKE_REQUIRED_FLAGS ${CMAKE_REQUIRED_FLAGS})
|
||||
string(REGEX REPLACE "[-+/ ]" "_" cname "${_name}")
|
||||
string(TOUPPER "${cname}" cname)
|
||||
foreach(flag ${_flags})
|
||||
string(REGEX REPLACE "^[-+/ ]+(.*)[-+/ ]*$" "\\1" flagname "${flag}")
|
||||
string(REGEX REPLACE "[-+/ ]" "_" flagname "${flagname}")
|
||||
string(TOUPPER "${flagname}" flagname)
|
||||
set(have_flag "HAVE_${cname}_${flagname}")
|
||||
set(CMAKE_REQUIRED_FLAGS "${flag}")
|
||||
check_c_source_compiles("int main() { return 0; }" ${have_flag})
|
||||
if(${have_flag})
|
||||
set(${_result} "${${_result}} ${flag}")
|
||||
endif(${have_flag})
|
||||
endforeach(flag)
|
||||
set(CMAKE_REQUIRED_FLAGS ${SAFE_CMAKE_REQUIRED_FLAGS})
|
||||
endmacro(append_c_compiler_flags)
|
||||
|
||||
macro(append_cxx_compiler_flags _flags _name _result)
|
||||
set(SAFE_CMAKE_REQUIRED_FLAGS ${CMAKE_REQUIRED_FLAGS})
|
||||
string(REGEX REPLACE "[-+/ ]" "_" cname "${_name}")
|
||||
string(TOUPPER "${cname}" cname)
|
||||
foreach(flag ${_flags})
|
||||
string(REGEX REPLACE "^[-+/ ]+(.*)[-+/ ]*$" "\\1" flagname "${flag}")
|
||||
string(REGEX REPLACE "[-+/ ]" "_" flagname "${flagname}")
|
||||
string(TOUPPER "${flagname}" flagname)
|
||||
set(have_flag "HAVE_${cname}_${flagname}")
|
||||
set(CMAKE_REQUIRED_FLAGS "${flag}")
|
||||
check_cxx_source_compiles("int main() { return 0; }" ${have_flag})
|
||||
if(${have_flag})
|
||||
set(${_result} "${${_result}} ${flag}")
|
||||
endif(${have_flag})
|
||||
endforeach(flag)
|
||||
set(CMAKE_REQUIRED_FLAGS ${SAFE_CMAKE_REQUIRED_FLAGS})
|
||||
endmacro(append_cxx_compiler_flags)
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
include(CheckCSourceCompiles)
|
||||
|
||||
macro(check_function_keywords _wordlist)
|
||||
set(${_result} "")
|
||||
foreach(flag ${_wordlist})
|
||||
string(REGEX REPLACE "[-+/ ()]" "_" flagname "${flag}")
|
||||
string(TOUPPER "${flagname}" flagname)
|
||||
set(have_flag "HAVE_${flagname}")
|
||||
check_c_source_compiles("${flag} void func(); void func() { } int main() { func(); return 0; }" ${have_flag})
|
||||
if(${have_flag} AND NOT ${_result})
|
||||
set(${_result} "${flag}")
|
||||
# break()
|
||||
endif(${have_flag} AND NOT ${_result})
|
||||
endforeach(flag)
|
||||
endmacro(check_function_keywords)
|
||||
|
|
@ -0,0 +1,109 @@
|
|||
## Checks for large file support ##
|
||||
include(CheckIncludeFile)
|
||||
include(CheckSymbolExists)
|
||||
include(CheckTypeSize)
|
||||
|
||||
macro(check_lfs _isenable)
|
||||
set(LFS_OFF_T "")
|
||||
set(LFS_FOPEN "")
|
||||
set(LFS_FSEEK "")
|
||||
set(LFS_FTELL "")
|
||||
set(LFS_PRID "")
|
||||
|
||||
if(${_isenable})
|
||||
set(SAFE_CMAKE_REQUIRED_DEFINITIONS "${CMAKE_REQUIRED_DEFINITIONS}")
|
||||
set(CMAKE_REQUIRED_DEFINITIONS ${CMAKE_REQUIRED_DEFINITIONS}
|
||||
-D_LARGEFILE_SOURCE -D_LARGE_FILES -D_FILE_OFFSET_BITS=64
|
||||
-D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS)
|
||||
|
||||
check_include_file("sys/types.h" HAVE_SYS_TYPES_H)
|
||||
check_include_file("inttypes.h" HAVE_INTTYPES_H)
|
||||
check_include_file("stddef.h" HAVE_STDDEF_H)
|
||||
check_include_file("stdint.h" HAVE_STDINT_H)
|
||||
|
||||
# LFS type1: 8 <= sizeof(off_t), fseeko, ftello
|
||||
check_type_size("off_t" SIZEOF_OFF_T)
|
||||
if(SIZEOF_OFF_T GREATER 7)
|
||||
check_symbol_exists("fseeko" "stdio.h" HAVE_FSEEKO)
|
||||
check_symbol_exists("ftello" "stdio.h" HAVE_FTELLO)
|
||||
if(HAVE_FSEEKO AND HAVE_FTELLO)
|
||||
set(LFS_OFF_T "off_t")
|
||||
set(LFS_FOPEN "fopen")
|
||||
set(LFS_FSEEK "fseeko")
|
||||
set(LFS_FTELL "ftello")
|
||||
check_symbol_exists("PRIdMAX" "inttypes.h" HAVE_PRIDMAX)
|
||||
if(HAVE_PRIDMAX)
|
||||
set(LFS_PRID "PRIdMAX")
|
||||
else(HAVE_PRIDMAX)
|
||||
check_type_size("long" SIZEOF_LONG)
|
||||
check_type_size("int" SIZEOF_INT)
|
||||
if(SIZEOF_OFF_T GREATER SIZEOF_LONG)
|
||||
set(LFS_PRID "\"lld\"")
|
||||
elseif(SIZEOF_LONG GREATER SIZEOF_INT)
|
||||
set(LFS_PRID "\"ld\"")
|
||||
else(SIZEOF_OFF_T GREATER SIZEOF_LONG)
|
||||
set(LFS_PRID "\"d\"")
|
||||
endif(SIZEOF_OFF_T GREATER SIZEOF_LONG)
|
||||
endif(HAVE_PRIDMAX)
|
||||
endif(HAVE_FSEEKO AND HAVE_FTELLO)
|
||||
endif(SIZEOF_OFF_T GREATER 7)
|
||||
|
||||
# LFS type2: 8 <= sizeof(off64_t), fopen64, fseeko64, ftello64
|
||||
if(NOT LFS_OFF_T)
|
||||
check_type_size("off64_t" SIZEOF_OFF64_T)
|
||||
if(SIZEOF_OFF64_T GREATER 7)
|
||||
check_symbol_exists("fopen64" "stdio.h" HAVE_FOPEN64)
|
||||
check_symbol_exists("fseeko64" "stdio.h" HAVE_FSEEKO64)
|
||||
check_symbol_exists("ftello64" "stdio.h" HAVE_FTELLO64)
|
||||
if(HAVE_FOPEN64 AND HAVE_FSEEKO64 AND HAVE_FTELLO64)
|
||||
set(LFS_OFF_T "off64_t")
|
||||
set(LFS_FOPEN "fopen64")
|
||||
set(LFS_FSEEK "fseeko64")
|
||||
set(LFS_FTELL "ftello64")
|
||||
check_symbol_exists("PRIdMAX" "inttypes.h" HAVE_PRIDMAX)
|
||||
if(HAVE_PRIDMAX)
|
||||
set(LFS_PRID "PRIdMAX")
|
||||
else(HAVE_PRIDMAX)
|
||||
check_type_size("long" SIZEOF_LONG)
|
||||
check_type_size("int" SIZEOF_INT)
|
||||
if(SIZEOF_OFF64_T GREATER SIZEOF_LONG)
|
||||
set(LFS_PRID "\"lld\"")
|
||||
elseif(SIZEOF_LONG GREATER SIZEOF_INT)
|
||||
set(LFS_PRID "\"ld\"")
|
||||
else(SIZEOF_OFF64_T GREATER SIZEOF_LONG)
|
||||
set(LFS_PRID "\"d\"")
|
||||
endif(SIZEOF_OFF64_T GREATER SIZEOF_LONG)
|
||||
endif(HAVE_PRIDMAX)
|
||||
endif(HAVE_FOPEN64 AND HAVE_FSEEKO64 AND HAVE_FTELLO64)
|
||||
endif(SIZEOF_OFF64_T GREATER 7)
|
||||
endif(NOT LFS_OFF_T)
|
||||
|
||||
# LFS type3: 8 <= sizeof(__int64), _fseeki64, _ftelli64
|
||||
if(NOT LFS_OFF_T)
|
||||
check_type_size("__int64" SIZEOF___INT64)
|
||||
if(SIZEOF___INT64 GREATER 7)
|
||||
check_symbol_exists("_fseeki64" "stdio.h" HAVE__FSEEKI64)
|
||||
check_symbol_exists("_ftelli64" "stdio.h" HAVE__FTELLI64)
|
||||
if(HAVE__FSEEKI64 AND HAVE__FTELLI64)
|
||||
set(LFS_OFF_T "__int64")
|
||||
set(LFS_FOPEN "fopen")
|
||||
set(LFS_FSEEK "_fseeki64")
|
||||
set(LFS_FTELL "_ftelli64")
|
||||
set(LFS_PRID "\"I64d\"")
|
||||
endif(HAVE__FSEEKI64 AND HAVE__FTELLI64)
|
||||
endif(SIZEOF___INT64 GREATER 7)
|
||||
endif(NOT LFS_OFF_T)
|
||||
|
||||
set(CMAKE_REQUIRED_DEFINITIONS "${SAFE_CMAKE_REQUIRED_DEFINITIONS}")
|
||||
endif(${_isenable})
|
||||
|
||||
if(NOT LFS_OFF_T)
|
||||
## not found
|
||||
set(LFS_OFF_T "long")
|
||||
set(LFS_FOPEN "fopen")
|
||||
set(LFS_FSEEK "fseek")
|
||||
set(LFS_FTELL "ftell")
|
||||
set(LFS_PRID "\"ld\"")
|
||||
endif(NOT LFS_OFF_T)
|
||||
|
||||
endmacro(check_lfs)
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
# If the cmake version includes cpack, use it
|
||||
IF(EXISTS "${CMAKE_ROOT}/Modules/CPack.cmake")
|
||||
SET(CPACK_PACKAGE_DESCRIPTION_SUMMARY "${PROJECT_DESCRIPTION}")
|
||||
SET(CPACK_PACKAGE_VENDOR "${PROJECT_VENDOR}")
|
||||
SET(CPACK_PACKAGE_DESCRIPTION_FILE "${CMAKE_CURRENT_SOURCE_DIR}/README.md")
|
||||
SET(CPACK_RESOURCE_FILE_LICENSE "${CMAKE_CURRENT_SOURCE_DIR}/LICENSE")
|
||||
SET(CPACK_PACKAGE_VERSION_MAJOR "${PROJECT_VERSION_MAJOR}")
|
||||
SET(CPACK_PACKAGE_VERSION_MINOR "${PROJECT_VERSION_MINOR}")
|
||||
SET(CPACK_PACKAGE_VERSION_PATCH "${PROJECT_VERSION_PATCH}")
|
||||
# SET(CPACK_PACKAGE_INSTALL_DIRECTORY "${PROJECT_NAME} ${PROJECT_VERSION}")
|
||||
SET(CPACK_SOURCE_PACKAGE_FILE_NAME "${PROJECT_NAME}-${PROJECT_VERSION_FULL}")
|
||||
|
||||
IF(NOT DEFINED CPACK_SYSTEM_NAME)
|
||||
SET(CPACK_SYSTEM_NAME "${CMAKE_SYSTEM_NAME}-${CMAKE_SYSTEM_PROCESSOR}")
|
||||
ENDIF(NOT DEFINED CPACK_SYSTEM_NAME)
|
||||
|
||||
IF(${CPACK_SYSTEM_NAME} MATCHES Windows)
|
||||
IF(CMAKE_CL_64)
|
||||
SET(CPACK_SYSTEM_NAME win64-${CMAKE_SYSTEM_PROCESSOR})
|
||||
ELSE(CMAKE_CL_64)
|
||||
SET(CPACK_SYSTEM_NAME win32-${CMAKE_SYSTEM_PROCESSOR})
|
||||
ENDIF(CMAKE_CL_64)
|
||||
ENDIF(${CPACK_SYSTEM_NAME} MATCHES Windows)
|
||||
|
||||
IF(NOT DEFINED CPACK_PACKAGE_FILE_NAME)
|
||||
SET(CPACK_PACKAGE_FILE_NAME "${CPACK_SOURCE_PACKAGE_FILE_NAME}-${CPACK_SYSTEM_NAME}")
|
||||
ENDIF(NOT DEFINED CPACK_PACKAGE_FILE_NAME)
|
||||
|
||||
SET(CPACK_PACKAGE_CONTACT "${PROJECT_CONTACT}")
|
||||
IF(UNIX)
|
||||
SET(CPACK_STRIP_FILES "")
|
||||
SET(CPACK_SOURCE_STRIP_FILES "")
|
||||
# SET(CPACK_PACKAGE_EXECUTABLES "ccmake" "CMake")
|
||||
ENDIF(UNIX)
|
||||
SET(CPACK_SOURCE_IGNORE_FILES "/CVS/" "/build/" "/\\\\.build/" "/\\\\.svn/" "~$")
|
||||
# include CPack model once all variables are set
|
||||
INCLUDE(CPack)
|
||||
ENDIF(EXISTS "${CMAKE_ROOT}/Modules/CPack.cmake")
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
IF(NOT EXISTS "@CMAKE_CURRENT_BINARY_DIR@/install_manifest.txt")
|
||||
MESSAGE(FATAL_ERROR "Cannot find install manifest: \"@CMAKE_CURRENT_BINARY_DIR@/install_manifest.txt\"")
|
||||
ENDIF(NOT EXISTS "@CMAKE_CURRENT_BINARY_DIR@/install_manifest.txt")
|
||||
|
||||
FILE(READ "@CMAKE_CURRENT_BINARY_DIR@/install_manifest.txt" files)
|
||||
STRING(REGEX REPLACE "\n" ";" files "${files}")
|
||||
|
||||
SET(NUM 0)
|
||||
FOREACH(file ${files})
|
||||
IF(EXISTS "$ENV{DESTDIR}${file}")
|
||||
MESSAGE(STATUS "Looking for \"$ENV{DESTDIR}${file}\" - found")
|
||||
SET(UNINSTALL_CHECK_${NUM} 1)
|
||||
ELSE(EXISTS "$ENV{DESTDIR}${file}")
|
||||
MESSAGE(STATUS "Looking for \"$ENV{DESTDIR}${file}\" - not found")
|
||||
SET(UNINSTALL_CHECK_${NUM} 0)
|
||||
ENDIF(EXISTS "$ENV{DESTDIR}${file}")
|
||||
MATH(EXPR NUM "1 + ${NUM}")
|
||||
ENDFOREACH(file)
|
||||
|
||||
SET(NUM 0)
|
||||
FOREACH(file ${files})
|
||||
IF(${UNINSTALL_CHECK_${NUM}})
|
||||
MESSAGE(STATUS "Uninstalling \"$ENV{DESTDIR}${file}\"")
|
||||
EXEC_PROGRAM(
|
||||
"@CMAKE_COMMAND@" ARGS "-E remove \"$ENV{DESTDIR}${file}\""
|
||||
OUTPUT_VARIABLE rm_out
|
||||
RETURN_VALUE rm_retval
|
||||
)
|
||||
IF(NOT "${rm_retval}" STREQUAL 0)
|
||||
MESSAGE(FATAL_ERROR "Problem when removing \"$ENV{DESTDIR}${file}\"")
|
||||
ENDIF(NOT "${rm_retval}" STREQUAL 0)
|
||||
ENDIF(${UNINSTALL_CHECK_${NUM}})
|
||||
MATH(EXPR NUM "1 + ${NUM}")
|
||||
ENDFOREACH(file)
|
||||
|
||||
FILE(REMOVE "@CMAKE_CURRENT_BINARY_DIR@/install_manifest.txt")
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
The MIT License (MIT)
|
||||
|
||||
Copyright (c) 2003 Yuta Mori All rights reserved.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
|
@ -0,0 +1,140 @@
|
|||
# libdivsufsort
|
||||
|
||||
libdivsufsort is a software library that implements a lightweight suffix array construction algorithm.
|
||||
|
||||
## News
|
||||
* 2015-03-21: The project has moved from [Google Code](http://code.google.com/p/libdivsufsort/) to [GitHub](https://github.com/y-256/libdivsufsort)
|
||||
|
||||
## Introduction
|
||||
This library provides a simple and an efficient C API to construct a suffix array and a Burrows-Wheeler transformed string from a given string over a constant-size alphabet.
|
||||
The algorithm runs in O(n log n) worst-case time using only 5n+O(1) bytes of memory space, where n is the length of
|
||||
the string.
|
||||
|
||||
## Build requirements
|
||||
* An ANSI C Compiler (e.g. GNU GCC)
|
||||
* [CMake](http://www.cmake.org/ "CMake") version 2.4.2 or newer
|
||||
* CMake-supported build tool
|
||||
|
||||
## Building on GNU/Linux
|
||||
1. Get the source code from GitHub. You can either
|
||||
* use git to clone the repository
|
||||
```
|
||||
git clone https://github.com/y-256/libdivsufsort.git
|
||||
```
|
||||
* or download a [zip file](../../archive/master.zip) directly
|
||||
2. Create a `build` directory in the package source directory.
|
||||
```shell
|
||||
$ cd libdivsufsort
|
||||
$ mkdir build
|
||||
$ cd build
|
||||
```
|
||||
3. Configure the package for your system.
|
||||
If you want to install to a different location, change the -DCMAKE_INSTALL_PREFIX option.
|
||||
```shell
|
||||
$ cmake -DCMAKE_BUILD_TYPE="Release" \
|
||||
-DCMAKE_INSTALL_PREFIX="/usr/local" ..
|
||||
```
|
||||
4. Compile the package.
|
||||
```shell
|
||||
$ make
|
||||
```
|
||||
5. (Optional) Install the library and header files.
|
||||
```shell
|
||||
$ sudo make install
|
||||
```
|
||||
|
||||
## API
|
||||
```c
|
||||
/* Data types */
|
||||
typedef int32_t saint_t;
|
||||
typedef int32_t saidx_t;
|
||||
typedef uint8_t sauchar_t;
|
||||
|
||||
/*
|
||||
* Constructs the suffix array of a given string.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param SA[0..n-1] The output array or suffixes.
|
||||
* @param n The length of the given string.
|
||||
* @return 0 if no error occurred, -1 or -2 otherwise.
|
||||
*/
|
||||
saint_t
|
||||
divsufsort(const sauchar_t *T, saidx_t *SA, saidx_t n);
|
||||
|
||||
/*
|
||||
* Constructs the burrows-wheeler transformed string of a given string.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param U[0..n-1] The output string. (can be T)
|
||||
* @param A[0..n-1] The temporary array. (can be NULL)
|
||||
* @param n The length of the given string.
|
||||
* @return The primary index if no error occurred, -1 or -2 otherwise.
|
||||
*/
|
||||
saidx_t
|
||||
divbwt(const sauchar_t *T, sauchar_t *U, saidx_t *A, saidx_t n);
|
||||
```
|
||||
|
||||
## Example Usage
|
||||
```c
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
#include <divsufsort.h>
|
||||
|
||||
int main() {
|
||||
// intput data
|
||||
char *Text = "abracadabra";
|
||||
int n = strlen(Text);
|
||||
int i, j;
|
||||
|
||||
// allocate
|
||||
int *SA = (int *)malloc(n * sizeof(int));
|
||||
|
||||
// sort
|
||||
divsufsort((unsigned char *)Text, SA, n);
|
||||
|
||||
// output
|
||||
for(i = 0; i < n; ++i) {
|
||||
printf("SA[%2d] = %2d: ", i, SA[i]);
|
||||
for(j = SA[i]; j < n; ++j) {
|
||||
printf("%c", Text[j]);
|
||||
}
|
||||
printf("$\n");
|
||||
}
|
||||
|
||||
// deallocate
|
||||
free(SA);
|
||||
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
See the [examples](examples) directory for a few other examples.
|
||||
|
||||
## Benchmarks
|
||||
See [Benchmarks](https://github.com/y-256/libdivsufsort/blob/wiki/SACA_Benchmarks.md) page for details.
|
||||
|
||||
## License
|
||||
libdivsufsort is released under the [MIT license](LICENSE "MIT license").
|
||||
> The MIT License (MIT)
|
||||
>
|
||||
> Copyright (c) 2003 Yuta Mori All rights reserved.
|
||||
>
|
||||
> Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
> of this software and associated documentation files (the "Software"), to deal
|
||||
> in the Software without restriction, including without limitation the rights
|
||||
> to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
> copies of the Software, and to permit persons to whom the Software is
|
||||
> furnished to do so, subject to the following conditions:
|
||||
>
|
||||
> The above copyright notice and this permission notice shall be included in all
|
||||
> copies or substantial portions of the Software.
|
||||
>
|
||||
> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
> IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
> FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
> AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
> LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
> OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
> SOFTWARE.
|
||||
|
||||
## Author
|
||||
* Yuta Mori
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
set(PROJECT_VERSION_MAJOR "2")
|
||||
set(PROJECT_VERSION_MINOR "0")
|
||||
set(PROJECT_VERSION_PATCH "2")
|
||||
set(PROJECT_VERSION_EXTRA "-1")
|
||||
set(PROJECT_VERSION "${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}")
|
||||
set(PROJECT_VERSION_FULL "${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}.${PROJECT_VERSION_PATCH}${PROJECT_VERSION_EXTRA}")
|
||||
|
||||
set(LIBRARY_VERSION "3.0.1")
|
||||
set(LIBRARY_SOVERSION "3")
|
||||
|
||||
## Git revision number ##
|
||||
if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/.git")
|
||||
execute_process(COMMAND git describe --tags HEAD
|
||||
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}"
|
||||
OUTPUT_VARIABLE GIT_DESCRIBE_TAGS ERROR_QUIET)
|
||||
if(GIT_DESCRIBE_TAGS)
|
||||
string(REGEX REPLACE "^v(.*)" "\\1" GIT_REVISION "${GIT_DESCRIBE_TAGS}")
|
||||
string(STRIP "${GIT_REVISION}" GIT_REVISION)
|
||||
if(GIT_REVISION)
|
||||
set(PROJECT_VERSION_FULL "${GIT_REVISION}")
|
||||
endif(GIT_REVISION)
|
||||
endif(GIT_DESCRIBE_TAGS)
|
||||
endif(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/.git")
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
## Add definitions ##
|
||||
add_definitions(-D_LARGEFILE_SOURCE -D_LARGE_FILES -D_FILE_OFFSET_BITS=64)
|
||||
|
||||
## Targets ##
|
||||
include_directories("${CMAKE_CURRENT_SOURCE_DIR}/../include"
|
||||
"${CMAKE_CURRENT_BINARY_DIR}/../include")
|
||||
link_directories("${CMAKE_CURRENT_BINARY_DIR}/../lib")
|
||||
foreach(src suftest mksary sasearch bwt unbwt)
|
||||
add_executable(${src} ${src}.c)
|
||||
target_link_libraries(${src} divsufsort)
|
||||
endforeach(src)
|
||||
|
|
@ -0,0 +1,220 @@
|
|||
/*
|
||||
* bwt.c for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#if HAVE_CONFIG_H
|
||||
# include "config.h"
|
||||
#endif
|
||||
#include <stdio.h>
|
||||
#if HAVE_STRING_H
|
||||
# include <string.h>
|
||||
#endif
|
||||
#if HAVE_STDLIB_H
|
||||
# include <stdlib.h>
|
||||
#endif
|
||||
#if HAVE_MEMORY_H
|
||||
# include <memory.h>
|
||||
#endif
|
||||
#if HAVE_STDDEF_H
|
||||
# include <stddef.h>
|
||||
#endif
|
||||
#if HAVE_STRINGS_H
|
||||
# include <strings.h>
|
||||
#endif
|
||||
#if HAVE_SYS_TYPES_H
|
||||
# include <sys/types.h>
|
||||
#endif
|
||||
#if HAVE_IO_H && HAVE_FCNTL_H
|
||||
# include <io.h>
|
||||
# include <fcntl.h>
|
||||
#endif
|
||||
#include <time.h>
|
||||
#include <divsufsort.h>
|
||||
#include "lfs.h"
|
||||
|
||||
|
||||
static
|
||||
size_t
|
||||
write_int(FILE *fp, saidx_t n) {
|
||||
unsigned char c[4];
|
||||
c[0] = (unsigned char)((n >> 0) & 0xff), c[1] = (unsigned char)((n >> 8) & 0xff),
|
||||
c[2] = (unsigned char)((n >> 16) & 0xff), c[3] = (unsigned char)((n >> 24) & 0xff);
|
||||
return fwrite(c, sizeof(unsigned char), 4, fp);
|
||||
}
|
||||
|
||||
static
|
||||
void
|
||||
print_help(const char *progname, int status) {
|
||||
fprintf(stderr,
|
||||
"bwt, a burrows-wheeler transform program, version %s.\n",
|
||||
divsufsort_version());
|
||||
fprintf(stderr, "usage: %s [-b num] INFILE OUTFILE\n", progname);
|
||||
fprintf(stderr, " -b num set block size to num MiB [1..512] (default: 32)\n\n");
|
||||
exit(status);
|
||||
}
|
||||
|
||||
int
|
||||
main(int argc, const char *argv[]) {
|
||||
FILE *fp, *ofp;
|
||||
const char *fname, *ofname;
|
||||
sauchar_t *T;
|
||||
saidx_t *SA;
|
||||
LFS_OFF_T n;
|
||||
size_t m;
|
||||
saidx_t pidx;
|
||||
clock_t start,finish;
|
||||
saint_t i, blocksize = 32, needclose = 3;
|
||||
|
||||
/* Check arguments. */
|
||||
if((argc == 1) ||
|
||||
(strcmp(argv[1], "-h") == 0) ||
|
||||
(strcmp(argv[1], "--help") == 0)) { print_help(argv[0], EXIT_SUCCESS); }
|
||||
if((argc != 3) && (argc != 5)) { print_help(argv[0], EXIT_FAILURE); }
|
||||
i = 1;
|
||||
if(argc == 5) {
|
||||
if(strcmp(argv[i], "-b") != 0) { print_help(argv[0], EXIT_FAILURE); }
|
||||
blocksize = atoi(argv[i + 1]);
|
||||
if(blocksize < 0) { blocksize = 1; }
|
||||
else if(512 < blocksize) { blocksize = 512; }
|
||||
i += 2;
|
||||
}
|
||||
blocksize <<= 20;
|
||||
|
||||
/* Open a file for reading. */
|
||||
if(strcmp(argv[i], "-") != 0) {
|
||||
#if HAVE_FOPEN_S
|
||||
if(fopen_s(&fp, fname = argv[i], "rb") != 0) {
|
||||
#else
|
||||
if((fp = LFS_FOPEN(fname = argv[i], "rb")) == NULL) {
|
||||
#endif
|
||||
fprintf(stderr, "%s: Cannot open file `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
} else {
|
||||
#if HAVE__SETMODE && HAVE__FILENO
|
||||
if(_setmode(_fileno(stdin), _O_BINARY) == -1) {
|
||||
fprintf(stderr, "%s: Cannot set mode: ", argv[0]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
#endif
|
||||
fp = stdin;
|
||||
fname = "stdin";
|
||||
needclose ^= 1;
|
||||
}
|
||||
i += 1;
|
||||
|
||||
/* Open a file for writing. */
|
||||
if(strcmp(argv[i], "-") != 0) {
|
||||
#if HAVE_FOPEN_S
|
||||
if(fopen_s(&ofp, ofname = argv[i], "wb") != 0) {
|
||||
#else
|
||||
if((ofp = LFS_FOPEN(ofname = argv[i], "wb")) == NULL) {
|
||||
#endif
|
||||
fprintf(stderr, "%s: Cannot open file `%s': ", argv[0], ofname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
} else {
|
||||
#if HAVE__SETMODE && HAVE__FILENO
|
||||
if(_setmode(_fileno(stdout), _O_BINARY) == -1) {
|
||||
fprintf(stderr, "%s: Cannot set mode: ", argv[0]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
#endif
|
||||
ofp = stdout;
|
||||
ofname = "stdout";
|
||||
needclose ^= 2;
|
||||
}
|
||||
|
||||
/* Get the file size. */
|
||||
if(LFS_FSEEK(fp, 0, SEEK_END) == 0) {
|
||||
n = LFS_FTELL(fp);
|
||||
rewind(fp);
|
||||
if(n < 0) {
|
||||
fprintf(stderr, "%s: Cannot ftell `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
if(0x20000000L < n) { n = 0x20000000L; }
|
||||
if((blocksize == 0) || (n < blocksize)) { blocksize = (saidx_t)n; }
|
||||
} else if(blocksize == 0) { blocksize = 32 << 20; }
|
||||
|
||||
/* Allocate 5blocksize bytes of memory. */
|
||||
T = (sauchar_t *)malloc(blocksize * sizeof(sauchar_t));
|
||||
SA = (saidx_t *)malloc(blocksize * sizeof(saidx_t));
|
||||
if((T == NULL) || (SA == NULL)) {
|
||||
fprintf(stderr, "%s: Cannot allocate memory.\n", argv[0]);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Write the blocksize. */
|
||||
if(write_int(ofp, blocksize) != 4) {
|
||||
fprintf(stderr, "%s: Cannot write to `%s': ", argv[0], ofname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
fprintf(stderr, " BWT (blocksize %" PRIdSAINT_T ") ... ", blocksize);
|
||||
start = clock();
|
||||
for(n = 0; 0 < (m = fread(T, sizeof(sauchar_t), blocksize, fp)); n += m) {
|
||||
/* Burrows-Wheeler Transform. */
|
||||
pidx = divbwt(T, T, SA, m);
|
||||
if(pidx < 0) {
|
||||
fprintf(stderr, "%s (bw_transform): %s.\n",
|
||||
argv[0],
|
||||
(pidx == -1) ? "Invalid arguments" : "Cannot allocate memory");
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Write the bwted data. */
|
||||
if((write_int(ofp, pidx) != 4) ||
|
||||
(fwrite(T, sizeof(sauchar_t), m, ofp) != m)) {
|
||||
fprintf(stderr, "%s: Cannot write to `%s': ", argv[0], ofname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
}
|
||||
if(ferror(fp)) {
|
||||
fprintf(stderr, "%s: Cannot read from `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
finish = clock();
|
||||
fprintf(stderr, "%" PRIdOFF_T " bytes: %.4f sec\n",
|
||||
n, (double)(finish - start) / (double)CLOCKS_PER_SEC);
|
||||
|
||||
/* Close files */
|
||||
if(needclose & 1) { fclose(fp); }
|
||||
if(needclose & 2) { fclose(ofp); }
|
||||
|
||||
/* Deallocate memory. */
|
||||
free(SA);
|
||||
free(T);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -0,0 +1,193 @@
|
|||
/*
|
||||
* mksary.c for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#if HAVE_CONFIG_H
|
||||
# include "config.h"
|
||||
#endif
|
||||
#include <stdio.h>
|
||||
#if HAVE_STRING_H
|
||||
# include <string.h>
|
||||
#endif
|
||||
#if HAVE_STDLIB_H
|
||||
# include <stdlib.h>
|
||||
#endif
|
||||
#if HAVE_MEMORY_H
|
||||
# include <memory.h>
|
||||
#endif
|
||||
#if HAVE_STDDEF_H
|
||||
# include <stddef.h>
|
||||
#endif
|
||||
#if HAVE_STRINGS_H
|
||||
# include <strings.h>
|
||||
#endif
|
||||
#if HAVE_SYS_TYPES_H
|
||||
# include <sys/types.h>
|
||||
#endif
|
||||
#if HAVE_IO_H && HAVE_FCNTL_H
|
||||
# include <io.h>
|
||||
# include <fcntl.h>
|
||||
#endif
|
||||
#include <time.h>
|
||||
#include <divsufsort.h>
|
||||
#include "lfs.h"
|
||||
|
||||
|
||||
static
|
||||
void
|
||||
print_help(const char *progname, int status) {
|
||||
fprintf(stderr,
|
||||
"mksary, a simple suffix array builder, version %s.\n",
|
||||
divsufsort_version());
|
||||
fprintf(stderr, "usage: %s INFILE OUTFILE\n\n", progname);
|
||||
exit(status);
|
||||
}
|
||||
|
||||
int
|
||||
main(int argc, const char *argv[]) {
|
||||
FILE *fp, *ofp;
|
||||
const char *fname, *ofname;
|
||||
sauchar_t *T;
|
||||
saidx_t *SA;
|
||||
LFS_OFF_T n;
|
||||
clock_t start, finish;
|
||||
saint_t needclose = 3;
|
||||
|
||||
/* Check arguments. */
|
||||
if((argc == 1) ||
|
||||
(strcmp(argv[1], "-h") == 0) ||
|
||||
(strcmp(argv[1], "--help") == 0)) { print_help(argv[0], EXIT_SUCCESS); }
|
||||
if(argc != 3) { print_help(argv[0], EXIT_FAILURE); }
|
||||
|
||||
/* Open a file for reading. */
|
||||
if(strcmp(argv[1], "-") != 0) {
|
||||
#if HAVE_FOPEN_S
|
||||
if(fopen_s(&fp, fname = argv[1], "rb") != 0) {
|
||||
#else
|
||||
if((fp = LFS_FOPEN(fname = argv[1], "rb")) == NULL) {
|
||||
#endif
|
||||
fprintf(stderr, "%s: Cannot open file `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
} else {
|
||||
#if HAVE__SETMODE && HAVE__FILENO
|
||||
if(_setmode(_fileno(stdin), _O_BINARY) == -1) {
|
||||
fprintf(stderr, "%s: Cannot set mode: ", argv[0]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
#endif
|
||||
fp = stdin;
|
||||
fname = "stdin";
|
||||
needclose ^= 1;
|
||||
}
|
||||
|
||||
/* Open a file for writing. */
|
||||
if(strcmp(argv[2], "-") != 0) {
|
||||
#if HAVE_FOPEN_S
|
||||
if(fopen_s(&ofp, ofname = argv[2], "wb") != 0) {
|
||||
#else
|
||||
if((ofp = LFS_FOPEN(ofname = argv[2], "wb")) == NULL) {
|
||||
#endif
|
||||
fprintf(stderr, "%s: Cannot open file `%s': ", argv[0], ofname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
} else {
|
||||
#if HAVE__SETMODE && HAVE__FILENO
|
||||
if(_setmode(_fileno(stdout), _O_BINARY) == -1) {
|
||||
fprintf(stderr, "%s: Cannot set mode: ", argv[0]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
#endif
|
||||
ofp = stdout;
|
||||
ofname = "stdout";
|
||||
needclose ^= 2;
|
||||
}
|
||||
|
||||
/* Get the file size. */
|
||||
if(LFS_FSEEK(fp, 0, SEEK_END) == 0) {
|
||||
n = LFS_FTELL(fp);
|
||||
rewind(fp);
|
||||
if(n < 0) {
|
||||
fprintf(stderr, "%s: Cannot ftell `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
if(0x7fffffff <= n) {
|
||||
fprintf(stderr, "%s: Input file `%s' is too big.\n", argv[0], fname);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
} else {
|
||||
fprintf(stderr, "%s: Cannot fseek `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Allocate 5blocksize bytes of memory. */
|
||||
T = (sauchar_t *)malloc((size_t)n * sizeof(sauchar_t));
|
||||
SA = (saidx_t *)malloc((size_t)n * sizeof(saidx_t));
|
||||
if((T == NULL) || (SA == NULL)) {
|
||||
fprintf(stderr, "%s: Cannot allocate memory.\n", argv[0]);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Read n bytes of data. */
|
||||
if(fread(T, sizeof(sauchar_t), (size_t)n, fp) != (size_t)n) {
|
||||
fprintf(stderr, "%s: %s `%s': ",
|
||||
argv[0],
|
||||
(ferror(fp) || !feof(fp)) ? "Cannot read from" : "Unexpected EOF in",
|
||||
fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
if(needclose & 1) { fclose(fp); }
|
||||
|
||||
/* Construct the suffix array. */
|
||||
fprintf(stderr, "%s: %" PRIdOFF_T " bytes ... ", fname, n);
|
||||
start = clock();
|
||||
if(divsufsort(T, SA, (saidx_t)n) != 0) {
|
||||
fprintf(stderr, "%s: Cannot allocate memory.\n", argv[0]);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
finish = clock();
|
||||
fprintf(stderr, "%.4f sec\n", (double)(finish - start) / (double)CLOCKS_PER_SEC);
|
||||
|
||||
/* Write the suffix array. */
|
||||
if(fwrite(SA, sizeof(saidx_t), (size_t)n, ofp) != (size_t)n) {
|
||||
fprintf(stderr, "%s: Cannot write to `%s': ", argv[0], ofname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
if(needclose & 2) { fclose(ofp); }
|
||||
|
||||
/* Deallocate memory. */
|
||||
free(SA);
|
||||
free(T);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -0,0 +1,165 @@
|
|||
/*
|
||||
* sasearch.c for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#if HAVE_CONFIG_H
|
||||
# include "config.h"
|
||||
#endif
|
||||
#include <stdio.h>
|
||||
#if HAVE_STRING_H
|
||||
# include <string.h>
|
||||
#endif
|
||||
#if HAVE_STDLIB_H
|
||||
# include <stdlib.h>
|
||||
#endif
|
||||
#if HAVE_MEMORY_H
|
||||
# include <memory.h>
|
||||
#endif
|
||||
#if HAVE_STDDEF_H
|
||||
# include <stddef.h>
|
||||
#endif
|
||||
#if HAVE_STRINGS_H
|
||||
# include <strings.h>
|
||||
#endif
|
||||
#if HAVE_SYS_TYPES_H
|
||||
# include <sys/types.h>
|
||||
#endif
|
||||
#if HAVE_IO_H && HAVE_FCNTL_H
|
||||
# include <io.h>
|
||||
# include <fcntl.h>
|
||||
#endif
|
||||
#include <divsufsort.h>
|
||||
#include "lfs.h"
|
||||
|
||||
|
||||
static
|
||||
void
|
||||
print_help(const char *progname, int status) {
|
||||
fprintf(stderr,
|
||||
"sasearch, a simple SA-based full-text search tool, version %s\n",
|
||||
divsufsort_version());
|
||||
fprintf(stderr, "usage: %s PATTERN FILE SAFILE\n\n", progname);
|
||||
exit(status);
|
||||
}
|
||||
|
||||
int
|
||||
main(int argc, const char *argv[]) {
|
||||
FILE *fp;
|
||||
const char *P;
|
||||
sauchar_t *T;
|
||||
saidx_t *SA;
|
||||
LFS_OFF_T n;
|
||||
size_t Psize;
|
||||
saidx_t i, size, left;
|
||||
|
||||
if((argc == 1) ||
|
||||
(strcmp(argv[1], "-h") == 0) ||
|
||||
(strcmp(argv[1], "--help") == 0)) { print_help(argv[0], EXIT_SUCCESS); }
|
||||
if(argc != 4) { print_help(argv[0], EXIT_FAILURE); }
|
||||
|
||||
P = argv[1];
|
||||
Psize = strlen(P);
|
||||
|
||||
/* Open a file for reading. */
|
||||
#if HAVE_FOPEN_S
|
||||
if(fopen_s(&fp, argv[2], "rb") != 0) {
|
||||
#else
|
||||
if((fp = LFS_FOPEN(argv[2], "rb")) == NULL) {
|
||||
#endif
|
||||
fprintf(stderr, "%s: Cannot open file `%s': ", argv[0], argv[2]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Get the file size. */
|
||||
if(LFS_FSEEK(fp, 0, SEEK_END) == 0) {
|
||||
n = LFS_FTELL(fp);
|
||||
rewind(fp);
|
||||
if(n < 0) {
|
||||
fprintf(stderr, "%s: Cannot ftell `%s': ", argv[0], argv[2]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
} else {
|
||||
fprintf(stderr, "%s: Cannot fseek `%s': ", argv[0], argv[2]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Allocate 5n bytes of memory. */
|
||||
T = (sauchar_t *)malloc((size_t)n * sizeof(sauchar_t));
|
||||
SA = (saidx_t *)malloc((size_t)n * sizeof(saidx_t));
|
||||
if((T == NULL) || (SA == NULL)) {
|
||||
fprintf(stderr, "%s: Cannot allocate memory.\n", argv[0]);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Read n bytes of data. */
|
||||
if(fread(T, sizeof(sauchar_t), (size_t)n, fp) != (size_t)n) {
|
||||
fprintf(stderr, "%s: %s `%s': ",
|
||||
argv[0],
|
||||
(ferror(fp) || !feof(fp)) ? "Cannot read from" : "Unexpected EOF in",
|
||||
argv[2]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
fclose(fp);
|
||||
|
||||
/* Open the SA file for reading. */
|
||||
#if HAVE_FOPEN_S
|
||||
if(fopen_s(&fp, argv[3], "rb") != 0) {
|
||||
#else
|
||||
if((fp = LFS_FOPEN(argv[3], "rb")) == NULL) {
|
||||
#endif
|
||||
fprintf(stderr, "%s: Cannot open file `%s': ", argv[0], argv[3]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Read n * sizeof(saidx_t) bytes of data. */
|
||||
if(fread(SA, sizeof(saidx_t), (size_t)n, fp) != (size_t)n) {
|
||||
fprintf(stderr, "%s: %s `%s': ",
|
||||
argv[0],
|
||||
(ferror(fp) || !feof(fp)) ? "Cannot read from" : "Unexpected EOF in",
|
||||
argv[3]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
fclose(fp);
|
||||
|
||||
/* Search and print */
|
||||
size = sa_search(T, (saidx_t)n,
|
||||
(const sauchar_t *)P, (saidx_t)Psize,
|
||||
SA, (saidx_t)n, &left);
|
||||
for(i = 0; i < size; ++i) {
|
||||
fprintf(stdout, "%" PRIdSAIDX_T "\n", SA[left + i]);
|
||||
}
|
||||
|
||||
/* Deallocate memory. */
|
||||
free(SA);
|
||||
free(T);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -0,0 +1,164 @@
|
|||
/*
|
||||
* suftest.c for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#if HAVE_CONFIG_H
|
||||
# include "config.h"
|
||||
#endif
|
||||
#include <stdio.h>
|
||||
#if HAVE_STRING_H
|
||||
# include <string.h>
|
||||
#endif
|
||||
#if HAVE_STDLIB_H
|
||||
# include <stdlib.h>
|
||||
#endif
|
||||
#if HAVE_MEMORY_H
|
||||
# include <memory.h>
|
||||
#endif
|
||||
#if HAVE_STDDEF_H
|
||||
# include <stddef.h>
|
||||
#endif
|
||||
#if HAVE_STRINGS_H
|
||||
# include <strings.h>
|
||||
#endif
|
||||
#if HAVE_SYS_TYPES_H
|
||||
# include <sys/types.h>
|
||||
#endif
|
||||
#if HAVE_IO_H && HAVE_FCNTL_H
|
||||
# include <io.h>
|
||||
# include <fcntl.h>
|
||||
#endif
|
||||
#include <time.h>
|
||||
#include <divsufsort.h>
|
||||
#include "lfs.h"
|
||||
|
||||
|
||||
static
|
||||
void
|
||||
print_help(const char *progname, int status) {
|
||||
fprintf(stderr,
|
||||
"suftest, a suffixsort tester, version %s.\n",
|
||||
divsufsort_version());
|
||||
fprintf(stderr, "usage: %s FILE\n\n", progname);
|
||||
exit(status);
|
||||
}
|
||||
|
||||
int
|
||||
main(int argc, const char *argv[]) {
|
||||
FILE *fp;
|
||||
const char *fname;
|
||||
sauchar_t *T;
|
||||
saidx_t *SA;
|
||||
LFS_OFF_T n;
|
||||
clock_t start, finish;
|
||||
saint_t needclose = 1;
|
||||
|
||||
/* Check arguments. */
|
||||
if((argc == 1) ||
|
||||
(strcmp(argv[1], "-h") == 0) ||
|
||||
(strcmp(argv[1], "--help") == 0)) { print_help(argv[0], EXIT_SUCCESS); }
|
||||
if(argc != 2) { print_help(argv[0], EXIT_FAILURE); }
|
||||
|
||||
/* Open a file for reading. */
|
||||
if(strcmp(argv[1], "-") != 0) {
|
||||
#if HAVE_FOPEN_S
|
||||
if(fopen_s(&fp, fname = argv[1], "rb") != 0) {
|
||||
#else
|
||||
if((fp = LFS_FOPEN(fname = argv[1], "rb")) == NULL) {
|
||||
#endif
|
||||
fprintf(stderr, "%s: Cannot open file `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
} else {
|
||||
#if HAVE__SETMODE && HAVE__FILENO
|
||||
if(_setmode(_fileno(stdin), _O_BINARY) == -1) {
|
||||
fprintf(stderr, "%s: Cannot set mode: ", argv[0]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
#endif
|
||||
fp = stdin;
|
||||
fname = "stdin";
|
||||
needclose = 0;
|
||||
}
|
||||
|
||||
/* Get the file size. */
|
||||
if(LFS_FSEEK(fp, 0, SEEK_END) == 0) {
|
||||
n = LFS_FTELL(fp);
|
||||
rewind(fp);
|
||||
if(n < 0) {
|
||||
fprintf(stderr, "%s: Cannot ftell `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
if(0x7fffffff <= n) {
|
||||
fprintf(stderr, "%s: Input file `%s' is too big.\n", argv[0], fname);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
} else {
|
||||
fprintf(stderr, "%s: Cannot fseek `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Allocate 5n bytes of memory. */
|
||||
T = (sauchar_t *)malloc((size_t)n * sizeof(sauchar_t));
|
||||
SA = (saidx_t *)malloc((size_t)n * sizeof(saidx_t));
|
||||
if((T == NULL) || (SA == NULL)) {
|
||||
fprintf(stderr, "%s: Cannot allocate memory.\n", argv[0]);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Read n bytes of data. */
|
||||
if(fread(T, sizeof(sauchar_t), (size_t)n, fp) != (size_t)n) {
|
||||
fprintf(stderr, "%s: %s `%s': ",
|
||||
argv[0],
|
||||
(ferror(fp) || !feof(fp)) ? "Cannot read from" : "Unexpected EOF in",
|
||||
argv[1]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
if(needclose & 1) { fclose(fp); }
|
||||
|
||||
/* Construct the suffix array. */
|
||||
fprintf(stderr, "%s: %" PRIdOFF_T " bytes ... ", fname, n);
|
||||
start = clock();
|
||||
if(divsufsort(T, SA, (saidx_t)n) != 0) {
|
||||
fprintf(stderr, "%s: Cannot allocate memory.\n", argv[0]);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
finish = clock();
|
||||
fprintf(stderr, "%.4f sec\n", (double)(finish - start) / (double)CLOCKS_PER_SEC);
|
||||
|
||||
/* Check the suffix array. */
|
||||
if(sufcheck(T, SA, (saidx_t)n, 1) != 0) { exit(EXIT_FAILURE); }
|
||||
|
||||
/* Deallocate memory. */
|
||||
free(SA);
|
||||
free(T);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -0,0 +1,207 @@
|
|||
/*
|
||||
* unbwt.c for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#if HAVE_CONFIG_H
|
||||
# include "config.h"
|
||||
#endif
|
||||
#include <stdio.h>
|
||||
#if HAVE_STRING_H
|
||||
# include <string.h>
|
||||
#endif
|
||||
#if HAVE_STDLIB_H
|
||||
# include <stdlib.h>
|
||||
#endif
|
||||
#if HAVE_MEMORY_H
|
||||
# include <memory.h>
|
||||
#endif
|
||||
#if HAVE_STDDEF_H
|
||||
# include <stddef.h>
|
||||
#endif
|
||||
#if HAVE_STRINGS_H
|
||||
# include <strings.h>
|
||||
#endif
|
||||
#if HAVE_SYS_TYPES_H
|
||||
# include <sys/types.h>
|
||||
#endif
|
||||
#if HAVE_IO_H && HAVE_FCNTL_H
|
||||
# include <io.h>
|
||||
# include <fcntl.h>
|
||||
#endif
|
||||
#include <time.h>
|
||||
#include <divsufsort.h>
|
||||
#include "lfs.h"
|
||||
|
||||
|
||||
static
|
||||
size_t
|
||||
read_int(FILE *fp, saidx_t *n) {
|
||||
unsigned char c[4];
|
||||
size_t m = fread(c, sizeof(unsigned char), 4, fp);
|
||||
if(m == 4) {
|
||||
*n = (c[0] << 0) | (c[1] << 8) |
|
||||
(c[2] << 16) | (c[3] << 24);
|
||||
}
|
||||
return m;
|
||||
}
|
||||
|
||||
static
|
||||
void
|
||||
print_help(const char *progname, int status) {
|
||||
fprintf(stderr,
|
||||
"unbwt, an inverse burrows-wheeler transform program, version %s.\n",
|
||||
divsufsort_version());
|
||||
fprintf(stderr, "usage: %s INFILE OUTFILE\n\n", progname);
|
||||
exit(status);
|
||||
}
|
||||
|
||||
int
|
||||
main(int argc, const char *argv[]) {
|
||||
FILE *fp, *ofp;
|
||||
const char *fname, *ofname;
|
||||
sauchar_t *T;
|
||||
saidx_t *A;
|
||||
LFS_OFF_T n;
|
||||
size_t m;
|
||||
saidx_t pidx;
|
||||
clock_t start, finish;
|
||||
saint_t err, blocksize, needclose = 3;
|
||||
|
||||
/* Check arguments. */
|
||||
if((argc == 1) ||
|
||||
(strcmp(argv[1], "-h") == 0) ||
|
||||
(strcmp(argv[1], "--help") == 0)) { print_help(argv[0], EXIT_SUCCESS); }
|
||||
if(argc != 3) { print_help(argv[0], EXIT_FAILURE); }
|
||||
|
||||
/* Open a file for reading. */
|
||||
if(strcmp(argv[1], "-") != 0) {
|
||||
#if HAVE_FOPEN_S
|
||||
if(fopen_s(&fp, fname = argv[1], "rb") != 0) {
|
||||
#else
|
||||
if((fp = LFS_FOPEN(fname = argv[1], "rb")) == NULL) {
|
||||
#endif
|
||||
fprintf(stderr, "%s: Cannot open file `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
} else {
|
||||
#if HAVE__SETMODE && HAVE__FILENO
|
||||
if(_setmode(_fileno(stdin), _O_BINARY) == -1) {
|
||||
fprintf(stderr, "%s: Cannot set mode: ", argv[0]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
#endif
|
||||
fp = stdin;
|
||||
fname = "stdin";
|
||||
needclose ^= 1;
|
||||
}
|
||||
|
||||
/* Open a file for writing. */
|
||||
if(strcmp(argv[2], "-") != 0) {
|
||||
#if HAVE_FOPEN_S
|
||||
if(fopen_s(&ofp, ofname = argv[2], "wb") != 0) {
|
||||
#else
|
||||
if((ofp = LFS_FOPEN(ofname = argv[2], "wb")) == NULL) {
|
||||
#endif
|
||||
fprintf(stderr, "%s: Cannot open file `%s': ", argv[0], ofname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
} else {
|
||||
#if HAVE__SETMODE && HAVE__FILENO
|
||||
if(_setmode(_fileno(stdout), _O_BINARY) == -1) {
|
||||
fprintf(stderr, "%s: Cannot set mode: ", argv[0]);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
#endif
|
||||
ofp = stdout;
|
||||
ofname = "stdout";
|
||||
needclose ^= 2;
|
||||
}
|
||||
|
||||
/* Read the blocksize. */
|
||||
if(read_int(fp, &blocksize) != 4) {
|
||||
fprintf(stderr, "%s: Cannot read from `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Allocate 5blocksize bytes of memory. */
|
||||
T = (sauchar_t *)malloc(blocksize * sizeof(sauchar_t));
|
||||
A = (saidx_t *)malloc(blocksize * sizeof(saidx_t));
|
||||
if((T == NULL) || (A == NULL)) {
|
||||
fprintf(stderr, "%s: Cannot allocate memory.\n", argv[0]);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
fprintf(stderr, "UnBWT (blocksize %" PRIdSAINT_T ") ... ", blocksize);
|
||||
start = clock();
|
||||
for(n = 0; (m = read_int(fp, &pidx)) != 0; n += m) {
|
||||
/* Read blocksize bytes of data. */
|
||||
if((m != 4) || ((m = fread(T, sizeof(sauchar_t), blocksize, fp)) == 0)) {
|
||||
fprintf(stderr, "%s: %s `%s': ",
|
||||
argv[0],
|
||||
(ferror(fp) || !feof(fp)) ? "Cannot read from" : "Unexpected EOF in",
|
||||
fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Inverse Burrows-Wheeler Transform. */
|
||||
if((err = inverse_bw_transform(T, T, A, m, pidx)) != 0) {
|
||||
fprintf(stderr, "%s (reverseBWT): %s.\n",
|
||||
argv[0],
|
||||
(err == -1) ? "Invalid data" : "Cannot allocate memory");
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* Write m bytes of data. */
|
||||
if(fwrite(T, sizeof(sauchar_t), m, ofp) != m) {
|
||||
fprintf(stderr, "%s: Cannot write to `%s': ", argv[0], ofname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
}
|
||||
if(ferror(fp)) {
|
||||
fprintf(stderr, "%s: Cannot read from `%s': ", argv[0], fname);
|
||||
perror(NULL);
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
finish = clock();
|
||||
fprintf(stderr, "%" PRIdOFF_T " bytes: %.4f sec\n",
|
||||
n, (double)(finish - start) / (double)CLOCKS_PER_SEC);
|
||||
|
||||
/* Close files */
|
||||
if(needclose & 1) { fclose(fp); }
|
||||
if(needclose & 2) { fclose(ofp); }
|
||||
|
||||
/* Deallocate memory. */
|
||||
free(A);
|
||||
free(T);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -0,0 +1,162 @@
|
|||
include(CheckIncludeFiles)
|
||||
include(CheckIncludeFile)
|
||||
include(CheckSymbolExists)
|
||||
include(CheckTypeSize)
|
||||
include(CheckFunctionKeywords)
|
||||
include(CheckLFS)
|
||||
|
||||
## Checks for header files ##
|
||||
check_include_file("inttypes.h" HAVE_INTTYPES_H)
|
||||
check_include_file("memory.h" HAVE_MEMORY_H)
|
||||
check_include_file("stddef.h" HAVE_STDDEF_H)
|
||||
check_include_file("stdint.h" HAVE_STDINT_H)
|
||||
check_include_file("stdlib.h" HAVE_STDLIB_H)
|
||||
check_include_file("string.h" HAVE_STRING_H)
|
||||
check_include_file("strings.h" HAVE_STRINGS_H)
|
||||
check_include_file("sys/types.h" HAVE_SYS_TYPES_H)
|
||||
if(HAVE_INTTYPES_H)
|
||||
set(INCFILE "#include <inttypes.h>")
|
||||
elseif(HAVE_STDINT_H)
|
||||
set(INCFILE "#include <stdint.h>")
|
||||
else(HAVE_INTTYPES_H)
|
||||
set(INCFILE "")
|
||||
endif(HAVE_INTTYPES_H)
|
||||
|
||||
## create configuration files from .cmake file ##
|
||||
if(BUILD_EXAMPLES)
|
||||
## Checks for WinIO ##
|
||||
if(WIN32)
|
||||
check_include_file("io.h" HAVE_IO_H)
|
||||
check_include_file("fcntl.h" HAVE_FCNTL_H)
|
||||
check_symbol_exists("_setmode" "io.h;fcntl.h" HAVE__SETMODE)
|
||||
if(NOT HAVE__SETMODE)
|
||||
check_symbol_exists("setmode" "io.h;fcntl.h" HAVE_SETMODE)
|
||||
endif(NOT HAVE__SETMODE)
|
||||
check_symbol_exists("_fileno" "stdio.h" HAVE__FILENO)
|
||||
check_symbol_exists("fopen_s" "stdio.h" HAVE_FOPEN_S)
|
||||
check_symbol_exists("_O_BINARY" "fcntl.h" HAVE__O_BINARY)
|
||||
endif(WIN32)
|
||||
|
||||
## Checks for large file support ##
|
||||
check_lfs(WITH_LFS)
|
||||
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/lfs.h.cmake" "${CMAKE_CURRENT_BINARY_DIR}/lfs.h" @ONLY)
|
||||
endif(BUILD_EXAMPLES)
|
||||
|
||||
## generate config.h ##
|
||||
check_function_keywords("inline;__inline;__inline__;__declspec(dllexport);__declspec(dllimport)")
|
||||
if(HAVE_INLINE)
|
||||
set(INLINE "inline")
|
||||
elseif(HAVE___INLINE)
|
||||
set(INLINE "__inline")
|
||||
elseif(HAVE___INLINE__)
|
||||
set(INLINE "__inline__")
|
||||
else(HAVE_INLINE)
|
||||
set(INLINE "")
|
||||
endif(HAVE_INLINE)
|
||||
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/config.h.cmake" "${CMAKE_CURRENT_BINARY_DIR}/config.h")
|
||||
|
||||
## Checks for types ##
|
||||
# sauchar_t (8bit)
|
||||
check_type_size("uint8_t" UINT8_T)
|
||||
if(HAVE_UINT8_T)
|
||||
set(SAUCHAR_TYPE "uint8_t")
|
||||
else(HAVE_UINT8_T)
|
||||
check_type_size("unsigned char" SIZEOF_UNSIGNED_CHAR)
|
||||
if("${SIZEOF_UNSIGNED_CHAR}" STREQUAL "1")
|
||||
set(SAUCHAR_TYPE "unsigned char")
|
||||
else("${SIZEOF_UNSIGNED_CHAR}" STREQUAL "1")
|
||||
message(FATAL_ERROR "Cannot find unsigned 8-bit integer type")
|
||||
endif("${SIZEOF_UNSIGNED_CHAR}" STREQUAL "1")
|
||||
endif(HAVE_UINT8_T)
|
||||
# saint_t (32bit)
|
||||
check_type_size("int32_t" INT32_T)
|
||||
if(HAVE_INT32_T)
|
||||
set(SAINT32_TYPE "int32_t")
|
||||
check_symbol_exists("PRId32" "inttypes.h" HAVE_PRID32)
|
||||
if(HAVE_PRID32)
|
||||
set(SAINT32_PRId "PRId32")
|
||||
else(HAVE_PRID32)
|
||||
set(SAINT32_PRId "\"d\"")
|
||||
endif(HAVE_PRID32)
|
||||
else(HAVE_INT32_T)
|
||||
check_type_size("int" SIZEOF_INT)
|
||||
check_type_size("long" SIZEOF_LONG)
|
||||
check_type_size("short" SIZEOF_SHORT)
|
||||
check_type_size("__int32" SIZEOF___INT32)
|
||||
if("${SIZEOF_INT}" STREQUAL "4")
|
||||
set(SAINT32_TYPE "int")
|
||||
set(SAINT32_PRId "\"d\"")
|
||||
elseif("${SIZEOF_LONG}" STREQUAL "4")
|
||||
set(SAINT32_TYPE "long")
|
||||
set(SAINT32_PRId "\"ld\"")
|
||||
elseif("${SIZEOF_SHORT}" STREQUAL "4")
|
||||
set(SAINT32_TYPE "short")
|
||||
set(SAINT32_PRId "\"d\"")
|
||||
elseif("${SIZEOF___INT32}" STREQUAL "4")
|
||||
set(SAINT32_TYPE "__int32")
|
||||
set(SAINT32_PRId "\"d\"")
|
||||
else("${SIZEOF_INT}" STREQUAL "4")
|
||||
message(FATAL_ERROR "Cannot find 32-bit integer type")
|
||||
endif("${SIZEOF_INT}" STREQUAL "4")
|
||||
endif(HAVE_INT32_T)
|
||||
# saint64_t (64bit)
|
||||
if(BUILD_DIVSUFSORT64)
|
||||
check_type_size("int64_t" INT64_T)
|
||||
if(HAVE_INT64_T)
|
||||
set(SAINT64_TYPE "int64_t")
|
||||
check_symbol_exists("PRId64" "inttypes.h" HAVE_PRID64)
|
||||
if(HAVE_PRID64)
|
||||
set(SAINT64_PRId "PRId64")
|
||||
else(HAVE_PRID64)
|
||||
set(SAINT64_PRId "\"lld\"")
|
||||
endif(HAVE_PRID64)
|
||||
else(HAVE_INT64_T)
|
||||
check_type_size("int" SIZEOF_INT)
|
||||
check_type_size("long" SIZEOF_LONG)
|
||||
check_type_size("long long" SIZEOF_LONG_LONG)
|
||||
check_type_size("__int64" SIZEOF___INT64)
|
||||
if("${SIZEOF_INT}" STREQUAL "8")
|
||||
set(SAINT64_TYPE "int")
|
||||
set(SAINT64_PRId "\"d\"")
|
||||
elseif("${SIZEOF_LONG}" STREQUAL "8")
|
||||
set(SAINT64_TYPE "long")
|
||||
set(SAINT64_PRId "\"ld\"")
|
||||
elseif("${SIZEOF_LONG_LONG}" STREQUAL "8")
|
||||
set(SAINT64_TYPE "long long")
|
||||
set(SAINT64_PRId "\"lld\"")
|
||||
elseif("${SIZEOF___INT64}" STREQUAL "8")
|
||||
set(SAINT64_TYPE "__int64")
|
||||
set(SAINT64_PRId "\"I64d\"")
|
||||
else("${SIZEOF_INT}" STREQUAL "8")
|
||||
message(SEND_ERROR "Cannot find 64-bit integer type")
|
||||
set(BUILD_DIVSUFSORT64 OFF)
|
||||
endif("${SIZEOF_INT}" STREQUAL "8")
|
||||
endif(HAVE_INT64_T)
|
||||
endif(BUILD_DIVSUFSORT64)
|
||||
|
||||
## generate divsufsort.h ##
|
||||
set(DIVSUFSORT_IMPORT "")
|
||||
set(DIVSUFSORT_EXPORT "")
|
||||
if(BUILD_SHARED_LIBS)
|
||||
if(HAVE___DECLSPEC_DLLIMPORT_)
|
||||
set(DIVSUFSORT_IMPORT "__declspec(dllimport)")
|
||||
endif(HAVE___DECLSPEC_DLLIMPORT_)
|
||||
if(HAVE___DECLSPEC_DLLEXPORT_)
|
||||
set(DIVSUFSORT_EXPORT "__declspec(dllexport)")
|
||||
endif(HAVE___DECLSPEC_DLLEXPORT_)
|
||||
endif(BUILD_SHARED_LIBS)
|
||||
set(W64BIT "")
|
||||
set(SAINDEX_TYPE "${SAINT32_TYPE}")
|
||||
set(SAINDEX_PRId "${SAINT32_PRId}")
|
||||
set(SAINT_PRId "${SAINT32_PRId}")
|
||||
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/divsufsort.h.cmake"
|
||||
"${CMAKE_CURRENT_BINARY_DIR}/divsufsort.h" @ONLY)
|
||||
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/divsufsort.h" DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
|
||||
if(BUILD_DIVSUFSORT64)
|
||||
set(W64BIT "64")
|
||||
set(SAINDEX_TYPE "${SAINT64_TYPE}")
|
||||
set(SAINDEX_PRId "${SAINT64_PRId}")
|
||||
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/divsufsort.h.cmake"
|
||||
"${CMAKE_CURRENT_BINARY_DIR}/divsufsort64.h" @ONLY)
|
||||
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/divsufsort64.h" DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
|
||||
endif(BUILD_DIVSUFSORT64)
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
#define HAVE_STRING_H 1
|
||||
#define HAVE_STDLIB_H 1
|
||||
#define HAVE_MEMORY_H 1
|
||||
#define HAVE_STDINT_H 1
|
||||
#define INLINE inline
|
||||
|
||||
#ifdef _MSC_VER
|
||||
#pragma warning( disable : 4244 )
|
||||
#endif /* _MSC_VER */
|
||||
|
|
@ -0,0 +1,81 @@
|
|||
/*
|
||||
* config.h for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#ifndef _CONFIG_H
|
||||
#define _CONFIG_H 1
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif /* __cplusplus */
|
||||
|
||||
/** Define to the version of this package. **/
|
||||
#cmakedefine PROJECT_VERSION_FULL "${PROJECT_VERSION_FULL}"
|
||||
|
||||
/** Define to 1 if you have the header files. **/
|
||||
#cmakedefine HAVE_INTTYPES_H 1
|
||||
#cmakedefine HAVE_STDDEF_H 1
|
||||
#cmakedefine HAVE_STDINT_H 1
|
||||
#cmakedefine HAVE_STDLIB_H 1
|
||||
#cmakedefine HAVE_STRING_H 1
|
||||
#cmakedefine HAVE_STRINGS_H 1
|
||||
#cmakedefine HAVE_MEMORY_H 1
|
||||
#cmakedefine HAVE_SYS_TYPES_H 1
|
||||
|
||||
/** for WinIO **/
|
||||
#cmakedefine HAVE_IO_H 1
|
||||
#cmakedefine HAVE_FCNTL_H 1
|
||||
#cmakedefine HAVE__SETMODE 1
|
||||
#cmakedefine HAVE_SETMODE 1
|
||||
#cmakedefine HAVE__FILENO 1
|
||||
#cmakedefine HAVE_FOPEN_S 1
|
||||
#cmakedefine HAVE__O_BINARY 1
|
||||
#ifndef HAVE__SETMODE
|
||||
# if HAVE_SETMODE
|
||||
# define _setmode setmode
|
||||
# define HAVE__SETMODE 1
|
||||
# endif
|
||||
# if HAVE__SETMODE && !HAVE__O_BINARY
|
||||
# define _O_BINARY 0
|
||||
# define HAVE__O_BINARY 1
|
||||
# endif
|
||||
#endif
|
||||
|
||||
/** for inline **/
|
||||
#ifndef INLINE
|
||||
# define INLINE @INLINE@
|
||||
#endif
|
||||
|
||||
/** for VC++ warning **/
|
||||
#ifdef _MSC_VER
|
||||
#pragma warning(disable: 4127)
|
||||
#endif
|
||||
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif /* __cplusplus */
|
||||
|
||||
#endif /* _CONFIG_H */
|
||||
|
|
@ -0,0 +1,168 @@
|
|||
/*
|
||||
* divsufsort.h for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#ifndef _DIVSUFSORT_H
|
||||
#define _DIVSUFSORT_H 1
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif /* __cplusplus */
|
||||
|
||||
#define DIVSUFSORT_API
|
||||
|
||||
/*- Datatypes -*/
|
||||
#ifndef SAUCHAR_T
|
||||
#define SAUCHAR_T
|
||||
typedef unsigned char sauchar_t;
|
||||
#endif /* SAUCHAR_T */
|
||||
#ifndef SAINT_T
|
||||
#define SAINT_T
|
||||
typedef int saint_t;
|
||||
#endif /* SAINT_T */
|
||||
#ifndef SAIDX_T
|
||||
#define SAIDX_T
|
||||
typedef int saidx_t;
|
||||
#endif /* SAIDX_T */
|
||||
#ifndef PRIdSAIDX_T
|
||||
#define PRIdSAIDX_T "d"
|
||||
#endif
|
||||
|
||||
/*- Prototypes -*/
|
||||
|
||||
/**
|
||||
* Constructs the suffix array of a given string.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param SA[0..n-1] The output array of suffixes.
|
||||
* @param n The length of the given string.
|
||||
* @return 0 if no error occurred, -1 or -2 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saint_t divsufsort(const sauchar_t *T, saidx_t *SA, saidx_t n);
|
||||
|
||||
#if 0
|
||||
/**
|
||||
* Constructs the burrows-wheeler transformed string of a given string.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param U[0..n-1] The output string. (can be T)
|
||||
* @param A[0..n-1] The temporary array. (can be NULL)
|
||||
* @param n The length of the given string.
|
||||
* @return The primary index if no error occurred, -1 or -2 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saidx_t
|
||||
divbwt(const sauchar_t *T, sauchar_t *U, saidx_t *A, saidx_t n);
|
||||
|
||||
/**
|
||||
* Returns the version of the divsufsort library.
|
||||
* @return The version number string.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
const char *
|
||||
divsufsort_version(void);
|
||||
|
||||
|
||||
/**
|
||||
* Constructs the burrows-wheeler transformed string of a given string and suffix array.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param U[0..n-1] The output string. (can be T)
|
||||
* @param SA[0..n-1] The suffix array. (can be NULL)
|
||||
* @param n The length of the given string.
|
||||
* @param idx The output primary index.
|
||||
* @return 0 if no error occurred, -1 or -2 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saint_t
|
||||
bw_transform(const sauchar_t *T, sauchar_t *U,
|
||||
saidx_t *SA /* can NULL */,
|
||||
saidx_t n, saidx_t *idx);
|
||||
|
||||
/**
|
||||
* Inverse BW-transforms a given BWTed string.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param U[0..n-1] The output string. (can be T)
|
||||
* @param A[0..n-1] The temporary array. (can be NULL)
|
||||
* @param n The length of the given string.
|
||||
* @param idx The primary index.
|
||||
* @return 0 if no error occurred, -1 or -2 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saint_t
|
||||
inverse_bw_transform(const sauchar_t *T, sauchar_t *U,
|
||||
saidx_t *A /* can NULL */,
|
||||
saidx_t n, saidx_t idx);
|
||||
|
||||
/**
|
||||
* Checks the correctness of a given suffix array.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param SA[0..n-1] The input suffix array.
|
||||
* @param n The length of the given string.
|
||||
* @param verbose The verbose mode.
|
||||
* @return 0 if no error occurred.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saint_t
|
||||
sufcheck(const sauchar_t *T, const saidx_t *SA, saidx_t n, saint_t verbose);
|
||||
|
||||
/**
|
||||
* Search for the pattern P in the string T.
|
||||
* @param T[0..Tsize-1] The input string.
|
||||
* @param Tsize The length of the given string.
|
||||
* @param P[0..Psize-1] The input pattern string.
|
||||
* @param Psize The length of the given pattern string.
|
||||
* @param SA[0..SAsize-1] The input suffix array.
|
||||
* @param SAsize The length of the given suffix array.
|
||||
* @param idx The output index.
|
||||
* @return The count of matches if no error occurred, -1 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saidx_t
|
||||
sa_search(const sauchar_t *T, saidx_t Tsize,
|
||||
const sauchar_t *P, saidx_t Psize,
|
||||
const saidx_t *SA, saidx_t SAsize,
|
||||
saidx_t *left);
|
||||
|
||||
/**
|
||||
* Search for the character c in the string T.
|
||||
* @param T[0..Tsize-1] The input string.
|
||||
* @param Tsize The length of the given string.
|
||||
* @param SA[0..SAsize-1] The input suffix array.
|
||||
* @param SAsize The length of the given suffix array.
|
||||
* @param c The input character.
|
||||
* @param idx The output index.
|
||||
* @return The count of matches if no error occurred, -1 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saidx_t
|
||||
sa_simplesearch(const sauchar_t *T, saidx_t Tsize,
|
||||
const saidx_t *SA, saidx_t SAsize,
|
||||
saint_t c, saidx_t *left);
|
||||
#endif
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif /* __cplusplus */
|
||||
|
||||
#endif /* _DIVSUFSORT_H */
|
||||
|
|
@ -0,0 +1,180 @@
|
|||
/*
|
||||
* divsufsort@W64BIT@.h for libdivsufsort@W64BIT@
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#ifndef _DIVSUFSORT@W64BIT@_H
|
||||
#define _DIVSUFSORT@W64BIT@_H 1
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif /* __cplusplus */
|
||||
|
||||
@INCFILE@
|
||||
|
||||
#ifndef DIVSUFSORT_API
|
||||
# ifdef DIVSUFSORT_BUILD_DLL
|
||||
# define DIVSUFSORT_API @DIVSUFSORT_EXPORT@
|
||||
# else
|
||||
# define DIVSUFSORT_API @DIVSUFSORT_IMPORT@
|
||||
# endif
|
||||
#endif
|
||||
|
||||
/*- Datatypes -*/
|
||||
#ifndef SAUCHAR_T
|
||||
#define SAUCHAR_T
|
||||
typedef @SAUCHAR_TYPE@ sauchar_t;
|
||||
#endif /* SAUCHAR_T */
|
||||
#ifndef SAINT_T
|
||||
#define SAINT_T
|
||||
typedef @SAINT32_TYPE@ saint_t;
|
||||
#endif /* SAINT_T */
|
||||
#ifndef SAIDX@W64BIT@_T
|
||||
#define SAIDX@W64BIT@_T
|
||||
typedef @SAINDEX_TYPE@ saidx@W64BIT@_t;
|
||||
#endif /* SAIDX@W64BIT@_T */
|
||||
#ifndef PRIdSAINT_T
|
||||
#define PRIdSAINT_T @SAINT_PRId@
|
||||
#endif /* PRIdSAINT_T */
|
||||
#ifndef PRIdSAIDX@W64BIT@_T
|
||||
#define PRIdSAIDX@W64BIT@_T @SAINDEX_PRId@
|
||||
#endif /* PRIdSAIDX@W64BIT@_T */
|
||||
|
||||
|
||||
/*- Prototypes -*/
|
||||
|
||||
/**
|
||||
* Constructs the suffix array of a given string.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param SA[0..n-1] The output array of suffixes.
|
||||
* @param n The length of the given string.
|
||||
* @return 0 if no error occurred, -1 or -2 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saint_t
|
||||
divsufsort@W64BIT@(const sauchar_t *T, saidx@W64BIT@_t *SA, saidx@W64BIT@_t n);
|
||||
|
||||
/**
|
||||
* Constructs the burrows-wheeler transformed string of a given string.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param U[0..n-1] The output string. (can be T)
|
||||
* @param A[0..n-1] The temporary array. (can be NULL)
|
||||
* @param n The length of the given string.
|
||||
* @return The primary index if no error occurred, -1 or -2 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saidx@W64BIT@_t
|
||||
divbwt@W64BIT@(const sauchar_t *T, sauchar_t *U, saidx@W64BIT@_t *A, saidx@W64BIT@_t n);
|
||||
|
||||
/**
|
||||
* Returns the version of the divsufsort library.
|
||||
* @return The version number string.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
const char *
|
||||
divsufsort@W64BIT@_version(void);
|
||||
|
||||
|
||||
/**
|
||||
* Constructs the burrows-wheeler transformed string of a given string and suffix array.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param U[0..n-1] The output string. (can be T)
|
||||
* @param SA[0..n-1] The suffix array. (can be NULL)
|
||||
* @param n The length of the given string.
|
||||
* @param idx The output primary index.
|
||||
* @return 0 if no error occurred, -1 or -2 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saint_t
|
||||
bw_transform@W64BIT@(const sauchar_t *T, sauchar_t *U,
|
||||
saidx@W64BIT@_t *SA /* can NULL */,
|
||||
saidx@W64BIT@_t n, saidx@W64BIT@_t *idx);
|
||||
|
||||
/**
|
||||
* Inverse BW-transforms a given BWTed string.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param U[0..n-1] The output string. (can be T)
|
||||
* @param A[0..n-1] The temporary array. (can be NULL)
|
||||
* @param n The length of the given string.
|
||||
* @param idx The primary index.
|
||||
* @return 0 if no error occurred, -1 or -2 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saint_t
|
||||
inverse_bw_transform@W64BIT@(const sauchar_t *T, sauchar_t *U,
|
||||
saidx@W64BIT@_t *A /* can NULL */,
|
||||
saidx@W64BIT@_t n, saidx@W64BIT@_t idx);
|
||||
|
||||
/**
|
||||
* Checks the correctness of a given suffix array.
|
||||
* @param T[0..n-1] The input string.
|
||||
* @param SA[0..n-1] The input suffix array.
|
||||
* @param n The length of the given string.
|
||||
* @param verbose The verbose mode.
|
||||
* @return 0 if no error occurred.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saint_t
|
||||
sufcheck@W64BIT@(const sauchar_t *T, const saidx@W64BIT@_t *SA, saidx@W64BIT@_t n, saint_t verbose);
|
||||
|
||||
/**
|
||||
* Search for the pattern P in the string T.
|
||||
* @param T[0..Tsize-1] The input string.
|
||||
* @param Tsize The length of the given string.
|
||||
* @param P[0..Psize-1] The input pattern string.
|
||||
* @param Psize The length of the given pattern string.
|
||||
* @param SA[0..SAsize-1] The input suffix array.
|
||||
* @param SAsize The length of the given suffix array.
|
||||
* @param idx The output index.
|
||||
* @return The count of matches if no error occurred, -1 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saidx@W64BIT@_t
|
||||
sa_search@W64BIT@(const sauchar_t *T, saidx@W64BIT@_t Tsize,
|
||||
const sauchar_t *P, saidx@W64BIT@_t Psize,
|
||||
const saidx@W64BIT@_t *SA, saidx@W64BIT@_t SAsize,
|
||||
saidx@W64BIT@_t *left);
|
||||
|
||||
/**
|
||||
* Search for the character c in the string T.
|
||||
* @param T[0..Tsize-1] The input string.
|
||||
* @param Tsize The length of the given string.
|
||||
* @param SA[0..SAsize-1] The input suffix array.
|
||||
* @param SAsize The length of the given suffix array.
|
||||
* @param c The input character.
|
||||
* @param idx The output index.
|
||||
* @return The count of matches if no error occurred, -1 otherwise.
|
||||
*/
|
||||
DIVSUFSORT_API
|
||||
saidx@W64BIT@_t
|
||||
sa_simplesearch@W64BIT@(const sauchar_t *T, saidx@W64BIT@_t Tsize,
|
||||
const saidx@W64BIT@_t *SA, saidx@W64BIT@_t SAsize,
|
||||
saint_t c, saidx@W64BIT@_t *left);
|
||||
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif /* __cplusplus */
|
||||
|
||||
#endif /* _DIVSUFSORT@W64BIT@_H */
|
||||
|
|
@ -0,0 +1,207 @@
|
|||
/*
|
||||
* divsufsort_private.h for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#ifndef _DIVSUFSORT_PRIVATE_H
|
||||
#define _DIVSUFSORT_PRIVATE_H 1
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif /* __cplusplus */
|
||||
|
||||
#if HAVE_CONFIG_H
|
||||
# include "config.h"
|
||||
#endif
|
||||
#include <assert.h>
|
||||
#include <stdio.h>
|
||||
#if HAVE_STRING_H
|
||||
# include <string.h>
|
||||
#endif
|
||||
#if HAVE_STDLIB_H
|
||||
# include <stdlib.h>
|
||||
#endif
|
||||
#if HAVE_MEMORY_H
|
||||
# include <memory.h>
|
||||
#endif
|
||||
#if HAVE_STDDEF_H
|
||||
# include <stddef.h>
|
||||
#endif
|
||||
#if HAVE_STRINGS_H
|
||||
# include <strings.h>
|
||||
#endif
|
||||
#if HAVE_INTTYPES_H
|
||||
# include <inttypes.h>
|
||||
#else
|
||||
# if HAVE_STDINT_H
|
||||
# include <stdint.h>
|
||||
# endif
|
||||
#endif
|
||||
#if defined(BUILD_DIVSUFSORT64)
|
||||
# include "divsufsort64.h"
|
||||
# ifndef SAIDX_T
|
||||
# define SAIDX_T
|
||||
# define saidx_t saidx64_t
|
||||
# endif /* SAIDX_T */
|
||||
# ifndef PRIdSAIDX_T
|
||||
# define PRIdSAIDX_T PRIdSAIDX64_T
|
||||
# endif /* PRIdSAIDX_T */
|
||||
# define divsufsort divsufsort64
|
||||
# define divbwt divbwt64
|
||||
# define divsufsort_version divsufsort64_version
|
||||
# define bw_transform bw_transform64
|
||||
# define inverse_bw_transform inverse_bw_transform64
|
||||
# define sufcheck sufcheck64
|
||||
# define sa_search sa_search64
|
||||
# define sa_simplesearch sa_simplesearch64
|
||||
# define sssort sssort64
|
||||
# define trsort trsort64
|
||||
#else
|
||||
# include "divsufsort.h"
|
||||
#endif
|
||||
|
||||
|
||||
/*- Constants -*/
|
||||
#if !defined(UINT8_MAX)
|
||||
# define UINT8_MAX (255)
|
||||
#endif /* UINT8_MAX */
|
||||
#if defined(ALPHABET_SIZE) && (ALPHABET_SIZE < 1)
|
||||
# undef ALPHABET_SIZE
|
||||
#endif
|
||||
#if !defined(ALPHABET_SIZE)
|
||||
# define ALPHABET_SIZE (UINT8_MAX + 1)
|
||||
#endif
|
||||
/* for divsufsort.c */
|
||||
#define BUCKET_A_SIZE (ALPHABET_SIZE)
|
||||
#define BUCKET_B_SIZE (ALPHABET_SIZE * ALPHABET_SIZE)
|
||||
/* for sssort.c */
|
||||
#if defined(SS_INSERTIONSORT_THRESHOLD)
|
||||
# if SS_INSERTIONSORT_THRESHOLD < 1
|
||||
# undef SS_INSERTIONSORT_THRESHOLD
|
||||
# define SS_INSERTIONSORT_THRESHOLD (1)
|
||||
# endif
|
||||
#else
|
||||
# define SS_INSERTIONSORT_THRESHOLD (8)
|
||||
#endif
|
||||
#if defined(SS_BLOCKSIZE)
|
||||
# if SS_BLOCKSIZE < 0
|
||||
# undef SS_BLOCKSIZE
|
||||
# define SS_BLOCKSIZE (0)
|
||||
# elif 32768 <= SS_BLOCKSIZE
|
||||
# undef SS_BLOCKSIZE
|
||||
# define SS_BLOCKSIZE (32767)
|
||||
# endif
|
||||
#else
|
||||
# define SS_BLOCKSIZE (1024)
|
||||
#endif
|
||||
/* minstacksize = log(SS_BLOCKSIZE) / log(3) * 2 */
|
||||
#if SS_BLOCKSIZE == 0
|
||||
# if defined(BUILD_DIVSUFSORT64)
|
||||
# define SS_MISORT_STACKSIZE (96)
|
||||
# else
|
||||
# define SS_MISORT_STACKSIZE (64)
|
||||
# endif
|
||||
#elif SS_BLOCKSIZE <= 4096
|
||||
# define SS_MISORT_STACKSIZE (16)
|
||||
#else
|
||||
# define SS_MISORT_STACKSIZE (24)
|
||||
#endif
|
||||
#if defined(BUILD_DIVSUFSORT64)
|
||||
# define SS_SMERGE_STACKSIZE (64)
|
||||
#else
|
||||
# define SS_SMERGE_STACKSIZE (32)
|
||||
#endif
|
||||
/* for trsort.c */
|
||||
#define TR_INSERTIONSORT_THRESHOLD (8)
|
||||
#if defined(BUILD_DIVSUFSORT64)
|
||||
# define TR_STACKSIZE (96)
|
||||
#else
|
||||
# define TR_STACKSIZE (64)
|
||||
#endif
|
||||
|
||||
|
||||
/*- Macros -*/
|
||||
#ifndef SWAP
|
||||
# define SWAP(_a, _b) do { t = (_a); (_a) = (_b); (_b) = t; } while(0)
|
||||
#endif /* SWAP */
|
||||
#ifndef MIN
|
||||
# define MIN(_a, _b) (((_a) < (_b)) ? (_a) : (_b))
|
||||
#endif /* MIN */
|
||||
#ifndef MAX
|
||||
# define MAX(_a, _b) (((_a) > (_b)) ? (_a) : (_b))
|
||||
#endif /* MAX */
|
||||
#define STACK_PUSH(_a, _b, _c, _d)\
|
||||
do {\
|
||||
assert(ssize < STACK_SIZE);\
|
||||
stack[ssize].a = (_a), stack[ssize].b = (_b),\
|
||||
stack[ssize].c = (_c), stack[ssize++].d = (_d);\
|
||||
} while(0)
|
||||
#define STACK_PUSH5(_a, _b, _c, _d, _e)\
|
||||
do {\
|
||||
assert(ssize < STACK_SIZE);\
|
||||
stack[ssize].a = (_a), stack[ssize].b = (_b),\
|
||||
stack[ssize].c = (_c), stack[ssize].d = (_d), stack[ssize++].e = (_e);\
|
||||
} while(0)
|
||||
#define STACK_POP(_a, _b, _c, _d)\
|
||||
do {\
|
||||
assert(0 <= ssize);\
|
||||
if(ssize == 0) { return; }\
|
||||
(_a) = stack[--ssize].a, (_b) = stack[ssize].b,\
|
||||
(_c) = stack[ssize].c, (_d) = stack[ssize].d;\
|
||||
} while(0)
|
||||
#define STACK_POP5(_a, _b, _c, _d, _e)\
|
||||
do {\
|
||||
assert(0 <= ssize);\
|
||||
if(ssize == 0) { return; }\
|
||||
(_a) = stack[--ssize].a, (_b) = stack[ssize].b,\
|
||||
(_c) = stack[ssize].c, (_d) = stack[ssize].d, (_e) = stack[ssize].e;\
|
||||
} while(0)
|
||||
/* for divsufsort.c */
|
||||
#define BUCKET_A(_c0) bucket_A[(_c0)]
|
||||
#if ALPHABET_SIZE == 256
|
||||
#define BUCKET_B(_c0, _c1) (bucket_B[((_c1) << 8) | (_c0)])
|
||||
#define BUCKET_BSTAR(_c0, _c1) (bucket_B[((_c0) << 8) | (_c1)])
|
||||
#else
|
||||
#define BUCKET_B(_c0, _c1) (bucket_B[(_c1) * ALPHABET_SIZE + (_c0)])
|
||||
#define BUCKET_BSTAR(_c0, _c1) (bucket_B[(_c0) * ALPHABET_SIZE + (_c1)])
|
||||
#endif
|
||||
|
||||
|
||||
/*- Private Prototypes -*/
|
||||
/* sssort.c */
|
||||
void
|
||||
sssort(const sauchar_t *Td, const saidx_t *PA,
|
||||
saidx_t *first, saidx_t *last,
|
||||
saidx_t *buf, saidx_t bufsize,
|
||||
saidx_t depth, saidx_t n, saint_t lastsuffix);
|
||||
/* trsort.c */
|
||||
void
|
||||
trsort(saidx_t *ISA, saidx_t *SA, saidx_t n, saidx_t depth);
|
||||
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif /* __cplusplus */
|
||||
|
||||
#endif /* _DIVSUFSORT_PRIVATE_H */
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
/*
|
||||
* lfs.h for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#ifndef _LFS_H
|
||||
#define _LFS_H 1
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif /* __cplusplus */
|
||||
|
||||
#ifndef __STRICT_ANSI__
|
||||
# define LFS_OFF_T @LFS_OFF_T@
|
||||
# define LFS_FOPEN @LFS_FOPEN@
|
||||
# define LFS_FTELL @LFS_FTELL@
|
||||
# define LFS_FSEEK @LFS_FSEEK@
|
||||
# define LFS_PRId @LFS_PRID@
|
||||
#else
|
||||
# define LFS_OFF_T long
|
||||
# define LFS_FOPEN fopen
|
||||
# define LFS_FTELL ftell
|
||||
# define LFS_FSEEK fseek
|
||||
# define LFS_PRId "ld"
|
||||
#endif
|
||||
#ifndef PRIdOFF_T
|
||||
# define PRIdOFF_T LFS_PRId
|
||||
#endif
|
||||
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif /* __cplusplus */
|
||||
|
||||
#endif /* _LFS_H */
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
include_directories("${CMAKE_CURRENT_SOURCE_DIR}/../include"
|
||||
"${CMAKE_CURRENT_BINARY_DIR}/../include")
|
||||
|
||||
set(divsufsort_SRCS divsufsort.c sssort.c trsort.c utils.c)
|
||||
|
||||
## libdivsufsort ##
|
||||
add_library(divsufsort ${divsufsort_SRCS})
|
||||
install(TARGETS divsufsort
|
||||
RUNTIME DESTINATION ${CMAKE_INSTALL_RUNTIMEDIR}
|
||||
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
|
||||
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR})
|
||||
set_target_properties(divsufsort PROPERTIES
|
||||
VERSION "${LIBRARY_VERSION}"
|
||||
SOVERSION "${LIBRARY_SOVERSION}"
|
||||
DEFINE_SYMBOL DIVSUFSORT_BUILD_DLL
|
||||
RUNTIME_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/../examples")
|
||||
|
||||
## libdivsufsort64 ##
|
||||
if(BUILD_DIVSUFSORT64)
|
||||
add_library(divsufsort64 ${divsufsort_SRCS})
|
||||
install(TARGETS divsufsort64
|
||||
RUNTIME DESTINATION ${CMAKE_INSTALL_RUNTIMEDIR}
|
||||
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
|
||||
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR})
|
||||
set_target_properties(divsufsort64 PROPERTIES
|
||||
VERSION "${LIBRARY_VERSION}"
|
||||
SOVERSION "${LIBRARY_SOVERSION}"
|
||||
DEFINE_SYMBOL DIVSUFSORT_BUILD_DLL
|
||||
COMPILE_FLAGS "-DBUILD_DIVSUFSORT64"
|
||||
RUNTIME_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/../examples")
|
||||
endif(BUILD_DIVSUFSORT64)
|
||||
|
|
@ -0,0 +1,401 @@
|
|||
/*
|
||||
* divsufsort.c for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#include "divsufsort_private.h"
|
||||
#ifdef _OPENMP
|
||||
# include <omp.h>
|
||||
#endif
|
||||
|
||||
|
||||
/*- Private Functions -*/
|
||||
|
||||
/* Sorts suffixes of type B*. */
|
||||
static
|
||||
saidx_t
|
||||
sort_typeBstar(const sauchar_t *T, saidx_t *SA,
|
||||
saidx_t *bucket_A, saidx_t *bucket_B,
|
||||
saidx_t n) {
|
||||
saidx_t *PAb, *ISAb, *buf;
|
||||
#ifdef _OPENMP
|
||||
saidx_t *curbuf;
|
||||
saidx_t l;
|
||||
#endif
|
||||
saidx_t i, j, k, t, m, bufsize;
|
||||
saint_t c0, c1;
|
||||
#ifdef _OPENMP
|
||||
saint_t d0, d1;
|
||||
int tmp;
|
||||
#endif
|
||||
|
||||
/* Initialize bucket arrays. */
|
||||
for(i = 0; i < BUCKET_A_SIZE; ++i) { bucket_A[i] = 0; }
|
||||
for(i = 0; i < BUCKET_B_SIZE; ++i) { bucket_B[i] = 0; }
|
||||
|
||||
/* Count the number of occurrences of the first one or two characters of each
|
||||
type A, B and B* suffix. Moreover, store the beginning position of all
|
||||
type B* suffixes into the array SA. */
|
||||
for(i = n - 1, m = n, c0 = T[n - 1]; 0 <= i;) {
|
||||
/* type A suffix. */
|
||||
do { ++BUCKET_A(c1 = c0); } while((0 <= --i) && ((c0 = T[i]) >= c1));
|
||||
if(0 <= i) {
|
||||
/* type B* suffix. */
|
||||
++BUCKET_BSTAR(c0, c1);
|
||||
SA[--m] = i;
|
||||
/* type B suffix. */
|
||||
for(--i, c1 = c0; (0 <= i) && ((c0 = T[i]) <= c1); --i, c1 = c0) {
|
||||
++BUCKET_B(c0, c1);
|
||||
}
|
||||
}
|
||||
}
|
||||
m = n - m;
|
||||
/*
|
||||
note:
|
||||
A type B* suffix is lexicographically smaller than a type B suffix that
|
||||
begins with the same first two characters.
|
||||
*/
|
||||
|
||||
/* Calculate the index of start/end point of each bucket. */
|
||||
for(c0 = 0, i = 0, j = 0; c0 < ALPHABET_SIZE; ++c0) {
|
||||
t = i + BUCKET_A(c0);
|
||||
BUCKET_A(c0) = i + j; /* start point */
|
||||
i = t + BUCKET_B(c0, c0);
|
||||
for(c1 = c0 + 1; c1 < ALPHABET_SIZE; ++c1) {
|
||||
j += BUCKET_BSTAR(c0, c1);
|
||||
BUCKET_BSTAR(c0, c1) = j; /* end point */
|
||||
i += BUCKET_B(c0, c1);
|
||||
}
|
||||
}
|
||||
|
||||
if(0 < m) {
|
||||
/* Sort the type B* suffixes by their first two characters. */
|
||||
PAb = SA + n - m; ISAb = SA + m;
|
||||
for(i = m - 2; 0 <= i; --i) {
|
||||
t = PAb[i], c0 = T[t], c1 = T[t + 1];
|
||||
SA[--BUCKET_BSTAR(c0, c1)] = i;
|
||||
}
|
||||
t = PAb[m - 1], c0 = T[t], c1 = T[t + 1];
|
||||
SA[--BUCKET_BSTAR(c0, c1)] = m - 1;
|
||||
|
||||
/* Sort the type B* substrings using sssort. */
|
||||
#ifdef _OPENMP
|
||||
tmp = omp_get_max_threads();
|
||||
buf = SA + m, bufsize = (n - (2 * m)) / tmp;
|
||||
c0 = ALPHABET_SIZE - 2, c1 = ALPHABET_SIZE - 1, j = m;
|
||||
#pragma omp parallel default(shared) private(curbuf, k, l, d0, d1, tmp)
|
||||
{
|
||||
tmp = omp_get_thread_num();
|
||||
curbuf = buf + tmp * bufsize;
|
||||
k = 0;
|
||||
for(;;) {
|
||||
#pragma omp critical(sssort_lock)
|
||||
{
|
||||
if(0 < (l = j)) {
|
||||
d0 = c0, d1 = c1;
|
||||
do {
|
||||
k = BUCKET_BSTAR(d0, d1);
|
||||
if(--d1 <= d0) {
|
||||
d1 = ALPHABET_SIZE - 1;
|
||||
if(--d0 < 0) { break; }
|
||||
}
|
||||
} while(((l - k) <= 1) && (0 < (l = k)));
|
||||
c0 = d0, c1 = d1, j = k;
|
||||
}
|
||||
}
|
||||
if(l == 0) { break; }
|
||||
sssort(T, PAb, SA + k, SA + l,
|
||||
curbuf, bufsize, 2, n, *(SA + k) == (m - 1));
|
||||
}
|
||||
}
|
||||
#else
|
||||
buf = SA + m, bufsize = n - (2 * m);
|
||||
for(c0 = ALPHABET_SIZE - 2, j = m; 0 < j; --c0) {
|
||||
for(c1 = ALPHABET_SIZE - 1; c0 < c1; j = i, --c1) {
|
||||
i = BUCKET_BSTAR(c0, c1);
|
||||
if(1 < (j - i)) {
|
||||
sssort(T, PAb, SA + i, SA + j,
|
||||
buf, bufsize, 2, n, *(SA + i) == (m - 1));
|
||||
}
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
/* Compute ranks of type B* substrings. */
|
||||
for(i = m - 1; 0 <= i; --i) {
|
||||
if(0 <= SA[i]) {
|
||||
j = i;
|
||||
do { ISAb[SA[i]] = i; } while((0 <= --i) && (0 <= SA[i]));
|
||||
SA[i + 1] = i - j;
|
||||
if(i <= 0) { break; }
|
||||
}
|
||||
j = i;
|
||||
do { ISAb[SA[i] = ~SA[i]] = j; } while(SA[--i] < 0);
|
||||
ISAb[SA[i]] = j;
|
||||
}
|
||||
|
||||
/* Construct the inverse suffix array of type B* suffixes using trsort. */
|
||||
trsort(ISAb, SA, m, 1);
|
||||
|
||||
/* Set the sorted order of tyoe B* suffixes. */
|
||||
for(i = n - 1, j = m, c0 = T[n - 1]; 0 <= i;) {
|
||||
for(--i, c1 = c0; (0 <= i) && ((c0 = T[i]) >= c1); --i, c1 = c0) { }
|
||||
if(0 <= i) {
|
||||
t = i;
|
||||
for(--i, c1 = c0; (0 <= i) && ((c0 = T[i]) <= c1); --i, c1 = c0) { }
|
||||
SA[ISAb[--j]] = ((t == 0) || (1 < (t - i))) ? t : ~t;
|
||||
}
|
||||
}
|
||||
|
||||
/* Calculate the index of start/end point of each bucket. */
|
||||
BUCKET_B(ALPHABET_SIZE - 1, ALPHABET_SIZE - 1) = n; /* end point */
|
||||
for(c0 = ALPHABET_SIZE - 2, k = m - 1; 0 <= c0; --c0) {
|
||||
i = BUCKET_A(c0 + 1) - 1;
|
||||
for(c1 = ALPHABET_SIZE - 1; c0 < c1; --c1) {
|
||||
t = i - BUCKET_B(c0, c1);
|
||||
BUCKET_B(c0, c1) = i; /* end point */
|
||||
|
||||
/* Move all type B* suffixes to the correct position. */
|
||||
for(i = t, j = BUCKET_BSTAR(c0, c1);
|
||||
j <= k;
|
||||
--i, --k) { SA[i] = SA[k]; }
|
||||
}
|
||||
BUCKET_BSTAR(c0, c0 + 1) = i - BUCKET_B(c0, c0) + 1; /* start point */
|
||||
BUCKET_B(c0, c0) = i; /* end point */
|
||||
}
|
||||
}
|
||||
|
||||
return m;
|
||||
}
|
||||
|
||||
/* Constructs the suffix array by using the sorted order of type B* suffixes. */
|
||||
static
|
||||
void
|
||||
construct_SA(const sauchar_t *T, saidx_t *SA,
|
||||
saidx_t *bucket_A, saidx_t *bucket_B,
|
||||
saidx_t n, saidx_t m) {
|
||||
saidx_t *i, *j, *k;
|
||||
saidx_t s;
|
||||
saint_t c0, c1, c2;
|
||||
|
||||
if(0 < m) {
|
||||
/* Construct the sorted order of type B suffixes by using
|
||||
the sorted order of type B* suffixes. */
|
||||
for(c1 = ALPHABET_SIZE - 2; 0 <= c1; --c1) {
|
||||
/* Scan the suffix array from right to left. */
|
||||
for(i = SA + BUCKET_BSTAR(c1, c1 + 1),
|
||||
j = SA + BUCKET_A(c1 + 1) - 1, k = NULL, c2 = -1;
|
||||
i <= j;
|
||||
--j) {
|
||||
if(0 < (s = *j)) {
|
||||
assert(T[s] == c1);
|
||||
assert(((s + 1) < n) && (T[s] <= T[s + 1]));
|
||||
assert(T[s - 1] <= T[s]);
|
||||
*j = ~s;
|
||||
c0 = T[--s];
|
||||
if((0 < s) && (T[s - 1] > c0)) { s = ~s; }
|
||||
if(c0 != c2) {
|
||||
if(0 <= c2) { BUCKET_B(c2, c1) = k - SA; }
|
||||
k = SA + BUCKET_B(c2 = c0, c1);
|
||||
}
|
||||
assert(k < j);
|
||||
*k-- = s;
|
||||
} else {
|
||||
assert(((s == 0) && (T[s] == c1)) || (s < 0));
|
||||
*j = ~s;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Construct the suffix array by using
|
||||
the sorted order of type B suffixes. */
|
||||
k = SA + BUCKET_A(c2 = T[n - 1]);
|
||||
*k++ = (T[n - 2] < c2) ? ~(n - 1) : (n - 1);
|
||||
/* Scan the suffix array from left to right. */
|
||||
for(i = SA, j = SA + n; i < j; ++i) {
|
||||
if(0 < (s = *i)) {
|
||||
assert(T[s - 1] >= T[s]);
|
||||
c0 = T[--s];
|
||||
if((s == 0) || (T[s - 1] < c0)) { s = ~s; }
|
||||
if(c0 != c2) {
|
||||
BUCKET_A(c2) = k - SA;
|
||||
k = SA + BUCKET_A(c2 = c0);
|
||||
}
|
||||
assert(i < k);
|
||||
*k++ = s;
|
||||
} else {
|
||||
assert(s < 0);
|
||||
*i = ~s;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#if 0
|
||||
/* Constructs the burrows-wheeler transformed string directly
|
||||
by using the sorted order of type B* suffixes. */
|
||||
static
|
||||
saidx_t
|
||||
construct_BWT(const sauchar_t *T, saidx_t *SA,
|
||||
saidx_t *bucket_A, saidx_t *bucket_B,
|
||||
saidx_t n, saidx_t m) {
|
||||
saidx_t *i, *j, *k, *orig;
|
||||
saidx_t s;
|
||||
saint_t c0, c1, c2;
|
||||
|
||||
if(0 < m) {
|
||||
/* Construct the sorted order of type B suffixes by using
|
||||
the sorted order of type B* suffixes. */
|
||||
for(c1 = ALPHABET_SIZE - 2; 0 <= c1; --c1) {
|
||||
/* Scan the suffix array from right to left. */
|
||||
for(i = SA + BUCKET_BSTAR(c1, c1 + 1),
|
||||
j = SA + BUCKET_A(c1 + 1) - 1, k = NULL, c2 = -1;
|
||||
i <= j;
|
||||
--j) {
|
||||
if(0 < (s = *j)) {
|
||||
assert(T[s] == c1);
|
||||
assert(((s + 1) < n) && (T[s] <= T[s + 1]));
|
||||
assert(T[s - 1] <= T[s]);
|
||||
c0 = T[--s];
|
||||
*j = ~((saidx_t)c0);
|
||||
if((0 < s) && (T[s - 1] > c0)) { s = ~s; }
|
||||
if(c0 != c2) {
|
||||
if(0 <= c2) { BUCKET_B(c2, c1) = k - SA; }
|
||||
k = SA + BUCKET_B(c2 = c0, c1);
|
||||
}
|
||||
assert(k < j);
|
||||
*k-- = s;
|
||||
} else if(s != 0) {
|
||||
*j = ~s;
|
||||
#ifndef NDEBUG
|
||||
} else {
|
||||
assert(T[s] == c1);
|
||||
#endif
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Construct the BWTed string by using
|
||||
the sorted order of type B suffixes. */
|
||||
k = SA + BUCKET_A(c2 = T[n - 1]);
|
||||
*k++ = (T[n - 2] < c2) ? ~((saidx_t)T[n - 2]) : (n - 1);
|
||||
/* Scan the suffix array from left to right. */
|
||||
for(i = SA, j = SA + n, orig = SA; i < j; ++i) {
|
||||
if(0 < (s = *i)) {
|
||||
assert(T[s - 1] >= T[s]);
|
||||
c0 = T[--s];
|
||||
*i = c0;
|
||||
if((0 < s) && (T[s - 1] < c0)) { s = ~((saidx_t)T[s - 1]); }
|
||||
if(c0 != c2) {
|
||||
BUCKET_A(c2) = k - SA;
|
||||
k = SA + BUCKET_A(c2 = c0);
|
||||
}
|
||||
assert(i < k);
|
||||
*k++ = s;
|
||||
} else if(s != 0) {
|
||||
*i = ~s;
|
||||
} else {
|
||||
orig = i;
|
||||
}
|
||||
}
|
||||
|
||||
return orig - SA;
|
||||
}
|
||||
#endif
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
/*- Function -*/
|
||||
|
||||
saint_t
|
||||
divsufsort(const sauchar_t *T, saidx_t *SA, saidx_t n) {
|
||||
saidx_t *bucket_A, *bucket_B;
|
||||
saidx_t m;
|
||||
saint_t err = 0;
|
||||
|
||||
/* Check arguments. */
|
||||
if((T == NULL) || (SA == NULL) || (n < 0)) { return -1; }
|
||||
else if(n == 0) { return 0; }
|
||||
else if(n == 1) { SA[0] = 0; return 0; }
|
||||
else if(n == 2) { m = (T[0] < T[1]); SA[m ^ 1] = 0, SA[m] = 1; return 0; }
|
||||
|
||||
bucket_A = (saidx_t *)malloc(BUCKET_A_SIZE * sizeof(saidx_t));
|
||||
bucket_B = (saidx_t *)malloc(BUCKET_B_SIZE * sizeof(saidx_t));
|
||||
|
||||
/* Suffixsort. */
|
||||
if((bucket_A != NULL) && (bucket_B != NULL)) {
|
||||
m = sort_typeBstar(T, SA, bucket_A, bucket_B, n);
|
||||
construct_SA(T, SA, bucket_A, bucket_B, n, m);
|
||||
} else {
|
||||
err = -2;
|
||||
}
|
||||
|
||||
free(bucket_B);
|
||||
free(bucket_A);
|
||||
|
||||
return err;
|
||||
}
|
||||
|
||||
#if 0
|
||||
saidx_t
|
||||
divbwt(const sauchar_t *T, sauchar_t *U, saidx_t *A, saidx_t n) {
|
||||
saidx_t *B;
|
||||
saidx_t *bucket_A, *bucket_B;
|
||||
saidx_t m, pidx, i;
|
||||
|
||||
/* Check arguments. */
|
||||
if((T == NULL) || (U == NULL) || (n < 0)) { return -1; }
|
||||
else if(n <= 1) { if(n == 1) { U[0] = T[0]; } return n; }
|
||||
|
||||
if((B = A) == NULL) { B = (saidx_t *)malloc((size_t)(n + 1) * sizeof(saidx_t)); }
|
||||
bucket_A = (saidx_t *)malloc(BUCKET_A_SIZE * sizeof(saidx_t));
|
||||
bucket_B = (saidx_t *)malloc(BUCKET_B_SIZE * sizeof(saidx_t));
|
||||
|
||||
/* Burrows-Wheeler Transform. */
|
||||
if((B != NULL) && (bucket_A != NULL) && (bucket_B != NULL)) {
|
||||
m = sort_typeBstar(T, B, bucket_A, bucket_B, n);
|
||||
pidx = construct_BWT(T, B, bucket_A, bucket_B, n, m);
|
||||
|
||||
/* Copy to output string. */
|
||||
U[0] = T[n - 1];
|
||||
for(i = 0; i < pidx; ++i) { U[i + 1] = (sauchar_t)B[i]; }
|
||||
for(i += 1; i < n; ++i) { U[i] = (sauchar_t)B[i]; }
|
||||
pidx += 1;
|
||||
} else {
|
||||
pidx = -2;
|
||||
}
|
||||
|
||||
free(bucket_B);
|
||||
free(bucket_A);
|
||||
if(A == NULL) { free(B); }
|
||||
|
||||
return pidx;
|
||||
}
|
||||
|
||||
const char *
|
||||
divsufsort_version(void) {
|
||||
return PROJECT_VERSION_FULL;
|
||||
}
|
||||
#endif
|
||||
|
|
@ -0,0 +1,815 @@
|
|||
/*
|
||||
* sssort.c for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#include "divsufsort_private.h"
|
||||
|
||||
|
||||
/*- Private Functions -*/
|
||||
|
||||
static const saint_t lg_table[256]= {
|
||||
-1,0,1,1,2,2,2,2,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,
|
||||
5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,
|
||||
6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,
|
||||
6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,
|
||||
7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,
|
||||
7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,
|
||||
7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,
|
||||
7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7
|
||||
};
|
||||
|
||||
#if (SS_BLOCKSIZE == 0) || (SS_INSERTIONSORT_THRESHOLD < SS_BLOCKSIZE)
|
||||
|
||||
static INLINE
|
||||
saint_t
|
||||
ss_ilg(saidx_t n) {
|
||||
#if SS_BLOCKSIZE == 0
|
||||
# if defined(BUILD_DIVSUFSORT64)
|
||||
return (n >> 32) ?
|
||||
((n >> 48) ?
|
||||
((n >> 56) ?
|
||||
56 + lg_table[(n >> 56) & 0xff] :
|
||||
48 + lg_table[(n >> 48) & 0xff]) :
|
||||
((n >> 40) ?
|
||||
40 + lg_table[(n >> 40) & 0xff] :
|
||||
32 + lg_table[(n >> 32) & 0xff])) :
|
||||
((n & 0xffff0000) ?
|
||||
((n & 0xff000000) ?
|
||||
24 + lg_table[(n >> 24) & 0xff] :
|
||||
16 + lg_table[(n >> 16) & 0xff]) :
|
||||
((n & 0x0000ff00) ?
|
||||
8 + lg_table[(n >> 8) & 0xff] :
|
||||
0 + lg_table[(n >> 0) & 0xff]));
|
||||
# else
|
||||
return (n & 0xffff0000) ?
|
||||
((n & 0xff000000) ?
|
||||
24 + lg_table[(n >> 24) & 0xff] :
|
||||
16 + lg_table[(n >> 16) & 0xff]) :
|
||||
((n & 0x0000ff00) ?
|
||||
8 + lg_table[(n >> 8) & 0xff] :
|
||||
0 + lg_table[(n >> 0) & 0xff]);
|
||||
# endif
|
||||
#elif SS_BLOCKSIZE < 256
|
||||
return lg_table[n];
|
||||
#else
|
||||
return (n & 0xff00) ?
|
||||
8 + lg_table[(n >> 8) & 0xff] :
|
||||
0 + lg_table[(n >> 0) & 0xff];
|
||||
#endif
|
||||
}
|
||||
|
||||
#endif /* (SS_BLOCKSIZE == 0) || (SS_INSERTIONSORT_THRESHOLD < SS_BLOCKSIZE) */
|
||||
|
||||
#if SS_BLOCKSIZE != 0
|
||||
|
||||
static const saint_t sqq_table[256] = {
|
||||
0, 16, 22, 27, 32, 35, 39, 42, 45, 48, 50, 53, 55, 57, 59, 61,
|
||||
64, 65, 67, 69, 71, 73, 75, 76, 78, 80, 81, 83, 84, 86, 87, 89,
|
||||
90, 91, 93, 94, 96, 97, 98, 99, 101, 102, 103, 104, 106, 107, 108, 109,
|
||||
110, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126,
|
||||
128, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142,
|
||||
143, 144, 144, 145, 146, 147, 148, 149, 150, 150, 151, 152, 153, 154, 155, 155,
|
||||
156, 157, 158, 159, 160, 160, 161, 162, 163, 163, 164, 165, 166, 167, 167, 168,
|
||||
169, 170, 170, 171, 172, 173, 173, 174, 175, 176, 176, 177, 178, 178, 179, 180,
|
||||
181, 181, 182, 183, 183, 184, 185, 185, 186, 187, 187, 188, 189, 189, 190, 191,
|
||||
192, 192, 193, 193, 194, 195, 195, 196, 197, 197, 198, 199, 199, 200, 201, 201,
|
||||
202, 203, 203, 204, 204, 205, 206, 206, 207, 208, 208, 209, 209, 210, 211, 211,
|
||||
212, 212, 213, 214, 214, 215, 215, 216, 217, 217, 218, 218, 219, 219, 220, 221,
|
||||
221, 222, 222, 223, 224, 224, 225, 225, 226, 226, 227, 227, 228, 229, 229, 230,
|
||||
230, 231, 231, 232, 232, 233, 234, 234, 235, 235, 236, 236, 237, 237, 238, 238,
|
||||
239, 240, 240, 241, 241, 242, 242, 243, 243, 244, 244, 245, 245, 246, 246, 247,
|
||||
247, 248, 248, 249, 249, 250, 250, 251, 251, 252, 252, 253, 253, 254, 254, 255
|
||||
};
|
||||
|
||||
static INLINE
|
||||
saidx_t
|
||||
ss_isqrt(saidx_t x) {
|
||||
saidx_t y, e;
|
||||
|
||||
if(x >= (SS_BLOCKSIZE * SS_BLOCKSIZE)) { return SS_BLOCKSIZE; }
|
||||
e = (x & 0xffff0000) ?
|
||||
((x & 0xff000000) ?
|
||||
24 + lg_table[(x >> 24) & 0xff] :
|
||||
16 + lg_table[(x >> 16) & 0xff]) :
|
||||
((x & 0x0000ff00) ?
|
||||
8 + lg_table[(x >> 8) & 0xff] :
|
||||
0 + lg_table[(x >> 0) & 0xff]);
|
||||
|
||||
if(e >= 16) {
|
||||
y = sqq_table[x >> ((e - 6) - (e & 1))] << ((e >> 1) - 7);
|
||||
if(e >= 24) { y = (y + 1 + x / y) >> 1; }
|
||||
y = (y + 1 + x / y) >> 1;
|
||||
} else if(e >= 8) {
|
||||
y = (sqq_table[x >> ((e - 6) - (e & 1))] >> (7 - (e >> 1))) + 1;
|
||||
} else {
|
||||
return sqq_table[x] >> 4;
|
||||
}
|
||||
|
||||
return (x < (y * y)) ? y - 1 : y;
|
||||
}
|
||||
|
||||
#endif /* SS_BLOCKSIZE != 0 */
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
/* Compares two suffixes. */
|
||||
static INLINE
|
||||
saint_t
|
||||
ss_compare(const sauchar_t *T,
|
||||
const saidx_t *p1, const saidx_t *p2,
|
||||
saidx_t depth) {
|
||||
const sauchar_t *U1, *U2, *U1n, *U2n;
|
||||
|
||||
for(U1 = T + depth + *p1,
|
||||
U2 = T + depth + *p2,
|
||||
U1n = T + *(p1 + 1) + 2,
|
||||
U2n = T + *(p2 + 1) + 2;
|
||||
(U1 < U1n) && (U2 < U2n) && (*U1 == *U2);
|
||||
++U1, ++U2) {
|
||||
}
|
||||
|
||||
return U1 < U1n ?
|
||||
(U2 < U2n ? *U1 - *U2 : 1) :
|
||||
(U2 < U2n ? -1 : 0);
|
||||
}
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
#if (SS_BLOCKSIZE != 1) && (SS_INSERTIONSORT_THRESHOLD != 1)
|
||||
|
||||
/* Insertionsort for small size groups */
|
||||
static
|
||||
void
|
||||
ss_insertionsort(const sauchar_t *T, const saidx_t *PA,
|
||||
saidx_t *first, saidx_t *last, saidx_t depth) {
|
||||
saidx_t *i, *j;
|
||||
saidx_t t;
|
||||
saint_t r;
|
||||
|
||||
for(i = last - 2; first <= i; --i) {
|
||||
for(t = *i, j = i + 1; 0 < (r = ss_compare(T, PA + t, PA + *j, depth));) {
|
||||
do { *(j - 1) = *j; } while((++j < last) && (*j < 0));
|
||||
if(last <= j) { break; }
|
||||
}
|
||||
if(r == 0) { *j = ~*j; }
|
||||
*(j - 1) = t;
|
||||
}
|
||||
}
|
||||
|
||||
#endif /* (SS_BLOCKSIZE != 1) && (SS_INSERTIONSORT_THRESHOLD != 1) */
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
#if (SS_BLOCKSIZE == 0) || (SS_INSERTIONSORT_THRESHOLD < SS_BLOCKSIZE)
|
||||
|
||||
static INLINE
|
||||
void
|
||||
ss_fixdown(const sauchar_t *Td, const saidx_t *PA,
|
||||
saidx_t *SA, saidx_t i, saidx_t size) {
|
||||
saidx_t j, k;
|
||||
saidx_t v;
|
||||
saint_t c, d, e;
|
||||
|
||||
for(v = SA[i], c = Td[PA[v]]; (j = 2 * i + 1) < size; SA[i] = SA[k], i = k) {
|
||||
d = Td[PA[SA[k = j++]]];
|
||||
if(d < (e = Td[PA[SA[j]]])) { k = j; d = e; }
|
||||
if(d <= c) { break; }
|
||||
}
|
||||
SA[i] = v;
|
||||
}
|
||||
|
||||
/* Simple top-down heapsort. */
|
||||
static
|
||||
void
|
||||
ss_heapsort(const sauchar_t *Td, const saidx_t *PA, saidx_t *SA, saidx_t size) {
|
||||
saidx_t i, m;
|
||||
saidx_t t;
|
||||
|
||||
m = size;
|
||||
if((size % 2) == 0) {
|
||||
m--;
|
||||
if(Td[PA[SA[m / 2]]] < Td[PA[SA[m]]]) { SWAP(SA[m], SA[m / 2]); }
|
||||
}
|
||||
|
||||
for(i = m / 2 - 1; 0 <= i; --i) { ss_fixdown(Td, PA, SA, i, m); }
|
||||
if((size % 2) == 0) { SWAP(SA[0], SA[m]); ss_fixdown(Td, PA, SA, 0, m); }
|
||||
for(i = m - 1; 0 < i; --i) {
|
||||
t = SA[0], SA[0] = SA[i];
|
||||
ss_fixdown(Td, PA, SA, 0, i);
|
||||
SA[i] = t;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
/* Returns the median of three elements. */
|
||||
static INLINE
|
||||
saidx_t *
|
||||
ss_median3(const sauchar_t *Td, const saidx_t *PA,
|
||||
saidx_t *v1, saidx_t *v2, saidx_t *v3) {
|
||||
saidx_t *t;
|
||||
if(Td[PA[*v1]] > Td[PA[*v2]]) { SWAP(v1, v2); }
|
||||
if(Td[PA[*v2]] > Td[PA[*v3]]) {
|
||||
if(Td[PA[*v1]] > Td[PA[*v3]]) { return v1; }
|
||||
else { return v3; }
|
||||
}
|
||||
return v2;
|
||||
}
|
||||
|
||||
/* Returns the median of five elements. */
|
||||
static INLINE
|
||||
saidx_t *
|
||||
ss_median5(const sauchar_t *Td, const saidx_t *PA,
|
||||
saidx_t *v1, saidx_t *v2, saidx_t *v3, saidx_t *v4, saidx_t *v5) {
|
||||
saidx_t *t;
|
||||
if(Td[PA[*v2]] > Td[PA[*v3]]) { SWAP(v2, v3); }
|
||||
if(Td[PA[*v4]] > Td[PA[*v5]]) { SWAP(v4, v5); }
|
||||
if(Td[PA[*v2]] > Td[PA[*v4]]) { SWAP(v2, v4); SWAP(v3, v5); }
|
||||
if(Td[PA[*v1]] > Td[PA[*v3]]) { SWAP(v1, v3); }
|
||||
if(Td[PA[*v1]] > Td[PA[*v4]]) { SWAP(v1, v4); SWAP(v3, v5); }
|
||||
if(Td[PA[*v3]] > Td[PA[*v4]]) { return v4; }
|
||||
return v3;
|
||||
}
|
||||
|
||||
/* Returns the pivot element. */
|
||||
static INLINE
|
||||
saidx_t *
|
||||
ss_pivot(const sauchar_t *Td, const saidx_t *PA, saidx_t *first, saidx_t *last) {
|
||||
saidx_t *middle;
|
||||
saidx_t t;
|
||||
|
||||
t = last - first;
|
||||
middle = first + t / 2;
|
||||
|
||||
if(t <= 512) {
|
||||
if(t <= 32) {
|
||||
return ss_median3(Td, PA, first, middle, last - 1);
|
||||
} else {
|
||||
t >>= 2;
|
||||
return ss_median5(Td, PA, first, first + t, middle, last - 1 - t, last - 1);
|
||||
}
|
||||
}
|
||||
t >>= 3;
|
||||
first = ss_median3(Td, PA, first, first + t, first + (t << 1));
|
||||
middle = ss_median3(Td, PA, middle - t, middle, middle + t);
|
||||
last = ss_median3(Td, PA, last - 1 - (t << 1), last - 1 - t, last - 1);
|
||||
return ss_median3(Td, PA, first, middle, last);
|
||||
}
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
/* Binary partition for substrings. */
|
||||
static INLINE
|
||||
saidx_t *
|
||||
ss_partition(const saidx_t *PA,
|
||||
saidx_t *first, saidx_t *last, saidx_t depth) {
|
||||
saidx_t *a, *b;
|
||||
saidx_t t;
|
||||
for(a = first - 1, b = last;;) {
|
||||
for(; (++a < b) && ((PA[*a] + depth) >= (PA[*a + 1] + 1));) { *a = ~*a; }
|
||||
for(; (a < --b) && ((PA[*b] + depth) < (PA[*b + 1] + 1));) { }
|
||||
if(b <= a) { break; }
|
||||
t = ~*b;
|
||||
*b = *a;
|
||||
*a = t;
|
||||
}
|
||||
if(first < a) { *first = ~*first; }
|
||||
return a;
|
||||
}
|
||||
|
||||
/* Multikey introsort for medium size groups. */
|
||||
static
|
||||
void
|
||||
ss_mintrosort(const sauchar_t *T, const saidx_t *PA,
|
||||
saidx_t *first, saidx_t *last,
|
||||
saidx_t depth) {
|
||||
#define STACK_SIZE SS_MISORT_STACKSIZE
|
||||
struct { saidx_t *a, *b, c; saint_t d; } stack[STACK_SIZE];
|
||||
const sauchar_t *Td;
|
||||
saidx_t *a, *b, *c, *d, *e, *f;
|
||||
saidx_t s, t;
|
||||
saint_t ssize;
|
||||
saint_t limit;
|
||||
saint_t v, x = 0;
|
||||
|
||||
for(ssize = 0, limit = ss_ilg(last - first);;) {
|
||||
|
||||
if((last - first) <= SS_INSERTIONSORT_THRESHOLD) {
|
||||
#if 1 < SS_INSERTIONSORT_THRESHOLD
|
||||
if(1 < (last - first)) { ss_insertionsort(T, PA, first, last, depth); }
|
||||
#endif
|
||||
STACK_POP(first, last, depth, limit);
|
||||
continue;
|
||||
}
|
||||
|
||||
Td = T + depth;
|
||||
if(limit-- == 0) { ss_heapsort(Td, PA, first, last - first); }
|
||||
if(limit < 0) {
|
||||
for(a = first + 1, v = Td[PA[*first]]; a < last; ++a) {
|
||||
if((x = Td[PA[*a]]) != v) {
|
||||
if(1 < (a - first)) { break; }
|
||||
v = x;
|
||||
first = a;
|
||||
}
|
||||
}
|
||||
if(Td[PA[*first] - 1] < v) {
|
||||
first = ss_partition(PA, first, a, depth);
|
||||
}
|
||||
if((a - first) <= (last - a)) {
|
||||
if(1 < (a - first)) {
|
||||
STACK_PUSH(a, last, depth, -1);
|
||||
last = a, depth += 1, limit = ss_ilg(a - first);
|
||||
} else {
|
||||
first = a, limit = -1;
|
||||
}
|
||||
} else {
|
||||
if(1 < (last - a)) {
|
||||
STACK_PUSH(first, a, depth + 1, ss_ilg(a - first));
|
||||
first = a, limit = -1;
|
||||
} else {
|
||||
last = a, depth += 1, limit = ss_ilg(a - first);
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
/* choose pivot */
|
||||
a = ss_pivot(Td, PA, first, last);
|
||||
v = Td[PA[*a]];
|
||||
SWAP(*first, *a);
|
||||
|
||||
/* partition */
|
||||
for(b = first; (++b < last) && ((x = Td[PA[*b]]) == v);) { }
|
||||
if(((a = b) < last) && (x < v)) {
|
||||
for(; (++b < last) && ((x = Td[PA[*b]]) <= v);) {
|
||||
if(x == v) { SWAP(*b, *a); ++a; }
|
||||
}
|
||||
}
|
||||
for(c = last; (b < --c) && ((x = Td[PA[*c]]) == v);) { }
|
||||
if((b < (d = c)) && (x > v)) {
|
||||
for(; (b < --c) && ((x = Td[PA[*c]]) >= v);) {
|
||||
if(x == v) { SWAP(*c, *d); --d; }
|
||||
}
|
||||
}
|
||||
for(; b < c;) {
|
||||
SWAP(*b, *c);
|
||||
for(; (++b < c) && ((x = Td[PA[*b]]) <= v);) {
|
||||
if(x == v) { SWAP(*b, *a); ++a; }
|
||||
}
|
||||
for(; (b < --c) && ((x = Td[PA[*c]]) >= v);) {
|
||||
if(x == v) { SWAP(*c, *d); --d; }
|
||||
}
|
||||
}
|
||||
|
||||
if(a <= d) {
|
||||
c = b - 1;
|
||||
|
||||
if((s = a - first) > (t = b - a)) { s = t; }
|
||||
for(e = first, f = b - s; 0 < s; --s, ++e, ++f) { SWAP(*e, *f); }
|
||||
if((s = d - c) > (t = last - d - 1)) { s = t; }
|
||||
for(e = b, f = last - s; 0 < s; --s, ++e, ++f) { SWAP(*e, *f); }
|
||||
|
||||
a = first + (b - a), c = last - (d - c);
|
||||
b = (v <= Td[PA[*a] - 1]) ? a : ss_partition(PA, a, c, depth);
|
||||
|
||||
if((a - first) <= (last - c)) {
|
||||
if((last - c) <= (c - b)) {
|
||||
STACK_PUSH(b, c, depth + 1, ss_ilg(c - b));
|
||||
STACK_PUSH(c, last, depth, limit);
|
||||
last = a;
|
||||
} else if((a - first) <= (c - b)) {
|
||||
STACK_PUSH(c, last, depth, limit);
|
||||
STACK_PUSH(b, c, depth + 1, ss_ilg(c - b));
|
||||
last = a;
|
||||
} else {
|
||||
STACK_PUSH(c, last, depth, limit);
|
||||
STACK_PUSH(first, a, depth, limit);
|
||||
first = b, last = c, depth += 1, limit = ss_ilg(c - b);
|
||||
}
|
||||
} else {
|
||||
if((a - first) <= (c - b)) {
|
||||
STACK_PUSH(b, c, depth + 1, ss_ilg(c - b));
|
||||
STACK_PUSH(first, a, depth, limit);
|
||||
first = c;
|
||||
} else if((last - c) <= (c - b)) {
|
||||
STACK_PUSH(first, a, depth, limit);
|
||||
STACK_PUSH(b, c, depth + 1, ss_ilg(c - b));
|
||||
first = c;
|
||||
} else {
|
||||
STACK_PUSH(first, a, depth, limit);
|
||||
STACK_PUSH(c, last, depth, limit);
|
||||
first = b, last = c, depth += 1, limit = ss_ilg(c - b);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
limit += 1;
|
||||
if(Td[PA[*first] - 1] < v) {
|
||||
first = ss_partition(PA, first, last, depth);
|
||||
limit = ss_ilg(last - first);
|
||||
}
|
||||
depth += 1;
|
||||
}
|
||||
}
|
||||
#undef STACK_SIZE
|
||||
}
|
||||
|
||||
#endif /* (SS_BLOCKSIZE == 0) || (SS_INSERTIONSORT_THRESHOLD < SS_BLOCKSIZE) */
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
#if SS_BLOCKSIZE != 0
|
||||
|
||||
static INLINE
|
||||
void
|
||||
ss_blockswap(saidx_t *a, saidx_t *b, saidx_t n) {
|
||||
saidx_t t;
|
||||
for(; 0 < n; --n, ++a, ++b) {
|
||||
t = *a, *a = *b, *b = t;
|
||||
}
|
||||
}
|
||||
|
||||
static INLINE
|
||||
void
|
||||
ss_rotate(saidx_t *first, saidx_t *middle, saidx_t *last) {
|
||||
saidx_t *a, *b, t;
|
||||
saidx_t l, r;
|
||||
l = middle - first, r = last - middle;
|
||||
for(; (0 < l) && (0 < r);) {
|
||||
if(l == r) { ss_blockswap(first, middle, l); break; }
|
||||
if(l < r) {
|
||||
a = last - 1, b = middle - 1;
|
||||
t = *a;
|
||||
do {
|
||||
*a-- = *b, *b-- = *a;
|
||||
if(b < first) {
|
||||
*a = t;
|
||||
last = a;
|
||||
if((r -= l + 1) <= l) { break; }
|
||||
a -= 1, b = middle - 1;
|
||||
t = *a;
|
||||
}
|
||||
} while(1);
|
||||
} else {
|
||||
a = first, b = middle;
|
||||
t = *a;
|
||||
do {
|
||||
*a++ = *b, *b++ = *a;
|
||||
if(last <= b) {
|
||||
*a = t;
|
||||
first = a + 1;
|
||||
if((l -= r + 1) <= r) { break; }
|
||||
a += 1, b = middle;
|
||||
t = *a;
|
||||
}
|
||||
} while(1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
static
|
||||
void
|
||||
ss_inplacemerge(const sauchar_t *T, const saidx_t *PA,
|
||||
saidx_t *first, saidx_t *middle, saidx_t *last,
|
||||
saidx_t depth) {
|
||||
const saidx_t *p;
|
||||
saidx_t *a, *b;
|
||||
saidx_t len, half;
|
||||
saint_t q, r;
|
||||
saint_t x;
|
||||
|
||||
for(;;) {
|
||||
if(*(last - 1) < 0) { x = 1; p = PA + ~*(last - 1); }
|
||||
else { x = 0; p = PA + *(last - 1); }
|
||||
for(a = first, len = middle - first, half = len >> 1, r = -1;
|
||||
0 < len;
|
||||
len = half, half >>= 1) {
|
||||
b = a + half;
|
||||
q = ss_compare(T, PA + ((0 <= *b) ? *b : ~*b), p, depth);
|
||||
if(q < 0) {
|
||||
a = b + 1;
|
||||
half -= (len & 1) ^ 1;
|
||||
} else {
|
||||
r = q;
|
||||
}
|
||||
}
|
||||
if(a < middle) {
|
||||
if(r == 0) { *a = ~*a; }
|
||||
ss_rotate(a, middle, last);
|
||||
last -= middle - a;
|
||||
middle = a;
|
||||
if(first == middle) { break; }
|
||||
}
|
||||
--last;
|
||||
if(x != 0) { while(*--last < 0) { } }
|
||||
if(middle == last) { break; }
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
/* Merge-forward with internal buffer. */
|
||||
static
|
||||
void
|
||||
ss_mergeforward(const sauchar_t *T, const saidx_t *PA,
|
||||
saidx_t *first, saidx_t *middle, saidx_t *last,
|
||||
saidx_t *buf, saidx_t depth) {
|
||||
saidx_t *a, *b, *c, *bufend;
|
||||
saidx_t t;
|
||||
saint_t r;
|
||||
|
||||
bufend = buf + (middle - first) - 1;
|
||||
ss_blockswap(buf, first, middle - first);
|
||||
|
||||
for(t = *(a = first), b = buf, c = middle;;) {
|
||||
r = ss_compare(T, PA + *b, PA + *c, depth);
|
||||
if(r < 0) {
|
||||
do {
|
||||
*a++ = *b;
|
||||
if(bufend <= b) { *bufend = t; return; }
|
||||
*b++ = *a;
|
||||
} while(*b < 0);
|
||||
} else if(r > 0) {
|
||||
do {
|
||||
*a++ = *c, *c++ = *a;
|
||||
if(last <= c) {
|
||||
while(b < bufend) { *a++ = *b, *b++ = *a; }
|
||||
*a = *b, *b = t;
|
||||
return;
|
||||
}
|
||||
} while(*c < 0);
|
||||
} else {
|
||||
*c = ~*c;
|
||||
do {
|
||||
*a++ = *b;
|
||||
if(bufend <= b) { *bufend = t; return; }
|
||||
*b++ = *a;
|
||||
} while(*b < 0);
|
||||
|
||||
do {
|
||||
*a++ = *c, *c++ = *a;
|
||||
if(last <= c) {
|
||||
while(b < bufend) { *a++ = *b, *b++ = *a; }
|
||||
*a = *b, *b = t;
|
||||
return;
|
||||
}
|
||||
} while(*c < 0);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Merge-backward with internal buffer. */
|
||||
static
|
||||
void
|
||||
ss_mergebackward(const sauchar_t *T, const saidx_t *PA,
|
||||
saidx_t *first, saidx_t *middle, saidx_t *last,
|
||||
saidx_t *buf, saidx_t depth) {
|
||||
const saidx_t *p1, *p2;
|
||||
saidx_t *a, *b, *c, *bufend;
|
||||
saidx_t t;
|
||||
saint_t r;
|
||||
saint_t x;
|
||||
|
||||
bufend = buf + (last - middle) - 1;
|
||||
ss_blockswap(buf, middle, last - middle);
|
||||
|
||||
x = 0;
|
||||
if(*bufend < 0) { p1 = PA + ~*bufend; x |= 1; }
|
||||
else { p1 = PA + *bufend; }
|
||||
if(*(middle - 1) < 0) { p2 = PA + ~*(middle - 1); x |= 2; }
|
||||
else { p2 = PA + *(middle - 1); }
|
||||
for(t = *(a = last - 1), b = bufend, c = middle - 1;;) {
|
||||
r = ss_compare(T, p1, p2, depth);
|
||||
if(0 < r) {
|
||||
if(x & 1) { do { *a-- = *b, *b-- = *a; } while(*b < 0); x ^= 1; }
|
||||
*a-- = *b;
|
||||
if(b <= buf) { *buf = t; break; }
|
||||
*b-- = *a;
|
||||
if(*b < 0) { p1 = PA + ~*b; x |= 1; }
|
||||
else { p1 = PA + *b; }
|
||||
} else if(r < 0) {
|
||||
if(x & 2) { do { *a-- = *c, *c-- = *a; } while(*c < 0); x ^= 2; }
|
||||
*a-- = *c, *c-- = *a;
|
||||
if(c < first) {
|
||||
while(buf < b) { *a-- = *b, *b-- = *a; }
|
||||
*a = *b, *b = t;
|
||||
break;
|
||||
}
|
||||
if(*c < 0) { p2 = PA + ~*c; x |= 2; }
|
||||
else { p2 = PA + *c; }
|
||||
} else {
|
||||
if(x & 1) { do { *a-- = *b, *b-- = *a; } while(*b < 0); x ^= 1; }
|
||||
*a-- = ~*b;
|
||||
if(b <= buf) { *buf = t; break; }
|
||||
*b-- = *a;
|
||||
if(x & 2) { do { *a-- = *c, *c-- = *a; } while(*c < 0); x ^= 2; }
|
||||
*a-- = *c, *c-- = *a;
|
||||
if(c < first) {
|
||||
while(buf < b) { *a-- = *b, *b-- = *a; }
|
||||
*a = *b, *b = t;
|
||||
break;
|
||||
}
|
||||
if(*b < 0) { p1 = PA + ~*b; x |= 1; }
|
||||
else { p1 = PA + *b; }
|
||||
if(*c < 0) { p2 = PA + ~*c; x |= 2; }
|
||||
else { p2 = PA + *c; }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* D&C based merge. */
|
||||
static
|
||||
void
|
||||
ss_swapmerge(const sauchar_t *T, const saidx_t *PA,
|
||||
saidx_t *first, saidx_t *middle, saidx_t *last,
|
||||
saidx_t *buf, saidx_t bufsize, saidx_t depth) {
|
||||
#define STACK_SIZE SS_SMERGE_STACKSIZE
|
||||
#define GETIDX(a) ((0 <= (a)) ? (a) : (~(a)))
|
||||
#define MERGE_CHECK(a, b, c)\
|
||||
do {\
|
||||
if(((c) & 1) ||\
|
||||
(((c) & 2) && (ss_compare(T, PA + GETIDX(*((a) - 1)), PA + *(a), depth) == 0))) {\
|
||||
*(a) = ~*(a);\
|
||||
}\
|
||||
if(((c) & 4) && ((ss_compare(T, PA + GETIDX(*((b) - 1)), PA + *(b), depth) == 0))) {\
|
||||
*(b) = ~*(b);\
|
||||
}\
|
||||
} while(0)
|
||||
struct { saidx_t *a, *b, *c; saint_t d; } stack[STACK_SIZE];
|
||||
saidx_t *l, *r, *lm, *rm;
|
||||
saidx_t m, len, half;
|
||||
saint_t ssize;
|
||||
saint_t check, next;
|
||||
|
||||
for(check = 0, ssize = 0;;) {
|
||||
if((last - middle) <= bufsize) {
|
||||
if((first < middle) && (middle < last)) {
|
||||
ss_mergebackward(T, PA, first, middle, last, buf, depth);
|
||||
}
|
||||
MERGE_CHECK(first, last, check);
|
||||
STACK_POP(first, middle, last, check);
|
||||
continue;
|
||||
}
|
||||
|
||||
if((middle - first) <= bufsize) {
|
||||
if(first < middle) {
|
||||
ss_mergeforward(T, PA, first, middle, last, buf, depth);
|
||||
}
|
||||
MERGE_CHECK(first, last, check);
|
||||
STACK_POP(first, middle, last, check);
|
||||
continue;
|
||||
}
|
||||
|
||||
for(m = 0, len = MIN(middle - first, last - middle), half = len >> 1;
|
||||
0 < len;
|
||||
len = half, half >>= 1) {
|
||||
if(ss_compare(T, PA + GETIDX(*(middle + m + half)),
|
||||
PA + GETIDX(*(middle - m - half - 1)), depth) < 0) {
|
||||
m += half + 1;
|
||||
half -= (len & 1) ^ 1;
|
||||
}
|
||||
}
|
||||
|
||||
if(0 < m) {
|
||||
lm = middle - m, rm = middle + m;
|
||||
ss_blockswap(lm, middle, m);
|
||||
l = r = middle, next = 0;
|
||||
if(rm < last) {
|
||||
if(*rm < 0) {
|
||||
*rm = ~*rm;
|
||||
if(first < lm) { for(; *--l < 0;) { } next |= 4; }
|
||||
next |= 1;
|
||||
} else if(first < lm) {
|
||||
for(; *r < 0; ++r) { }
|
||||
next |= 2;
|
||||
}
|
||||
}
|
||||
|
||||
if((l - first) <= (last - r)) {
|
||||
STACK_PUSH(r, rm, last, (next & 3) | (check & 4));
|
||||
middle = lm, last = l, check = (check & 3) | (next & 4);
|
||||
} else {
|
||||
if((next & 2) && (r == middle)) { next ^= 6; }
|
||||
STACK_PUSH(first, lm, l, (check & 3) | (next & 4));
|
||||
first = r, middle = rm, check = (next & 3) | (check & 4);
|
||||
}
|
||||
} else {
|
||||
if(ss_compare(T, PA + GETIDX(*(middle - 1)), PA + *middle, depth) == 0) {
|
||||
*middle = ~*middle;
|
||||
}
|
||||
MERGE_CHECK(first, last, check);
|
||||
STACK_POP(first, middle, last, check);
|
||||
}
|
||||
}
|
||||
#undef STACK_SIZE
|
||||
}
|
||||
|
||||
#endif /* SS_BLOCKSIZE != 0 */
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
/*- Function -*/
|
||||
|
||||
/* Substring sort */
|
||||
void
|
||||
sssort(const sauchar_t *T, const saidx_t *PA,
|
||||
saidx_t *first, saidx_t *last,
|
||||
saidx_t *buf, saidx_t bufsize,
|
||||
saidx_t depth, saidx_t n, saint_t lastsuffix) {
|
||||
saidx_t *a;
|
||||
#if SS_BLOCKSIZE != 0
|
||||
saidx_t *b, *middle, *curbuf;
|
||||
saidx_t j, k, curbufsize, limit;
|
||||
#endif
|
||||
saidx_t i;
|
||||
|
||||
if(lastsuffix != 0) { ++first; }
|
||||
|
||||
#if SS_BLOCKSIZE == 0
|
||||
ss_mintrosort(T, PA, first, last, depth);
|
||||
#else
|
||||
if((bufsize < SS_BLOCKSIZE) &&
|
||||
(bufsize < (last - first)) &&
|
||||
(bufsize < (limit = ss_isqrt(last - first)))) {
|
||||
if(SS_BLOCKSIZE < limit) { limit = SS_BLOCKSIZE; }
|
||||
buf = middle = last - limit, bufsize = limit;
|
||||
} else {
|
||||
middle = last, limit = 0;
|
||||
}
|
||||
for(a = first, i = 0; SS_BLOCKSIZE < (middle - a); a += SS_BLOCKSIZE, ++i) {
|
||||
#if SS_INSERTIONSORT_THRESHOLD < SS_BLOCKSIZE
|
||||
ss_mintrosort(T, PA, a, a + SS_BLOCKSIZE, depth);
|
||||
#elif 1 < SS_BLOCKSIZE
|
||||
ss_insertionsort(T, PA, a, a + SS_BLOCKSIZE, depth);
|
||||
#endif
|
||||
curbufsize = last - (a + SS_BLOCKSIZE);
|
||||
curbuf = a + SS_BLOCKSIZE;
|
||||
if(curbufsize <= bufsize) { curbufsize = bufsize, curbuf = buf; }
|
||||
for(b = a, k = SS_BLOCKSIZE, j = i; j & 1; b -= k, k <<= 1, j >>= 1) {
|
||||
ss_swapmerge(T, PA, b - k, b, b + k, curbuf, curbufsize, depth);
|
||||
}
|
||||
}
|
||||
#if SS_INSERTIONSORT_THRESHOLD < SS_BLOCKSIZE
|
||||
ss_mintrosort(T, PA, a, middle, depth);
|
||||
#elif 1 < SS_BLOCKSIZE
|
||||
ss_insertionsort(T, PA, a, middle, depth);
|
||||
#endif
|
||||
for(k = SS_BLOCKSIZE; i != 0; k <<= 1, i >>= 1) {
|
||||
if(i & 1) {
|
||||
ss_swapmerge(T, PA, a - k, a, middle, buf, bufsize, depth);
|
||||
a -= k;
|
||||
}
|
||||
}
|
||||
if(limit != 0) {
|
||||
#if SS_INSERTIONSORT_THRESHOLD < SS_BLOCKSIZE
|
||||
ss_mintrosort(T, PA, middle, last, depth);
|
||||
#elif 1 < SS_BLOCKSIZE
|
||||
ss_insertionsort(T, PA, middle, last, depth);
|
||||
#endif
|
||||
ss_inplacemerge(T, PA, first, middle, last, depth);
|
||||
}
|
||||
#endif
|
||||
|
||||
if(lastsuffix != 0) {
|
||||
/* Insert last type B* suffix. */
|
||||
saidx_t PAi[2]; PAi[0] = PA[*(first - 1)], PAi[1] = n - 2;
|
||||
for(a = first, i = *(first - 1);
|
||||
(a < last) && ((*a < 0) || (0 < ss_compare(T, &(PAi[0]), PA + *a, depth)));
|
||||
++a) {
|
||||
*(a - 1) = *a;
|
||||
}
|
||||
*(a - 1) = i;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,586 @@
|
|||
/*
|
||||
* trsort.c for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#include "divsufsort_private.h"
|
||||
|
||||
|
||||
/*- Private Functions -*/
|
||||
|
||||
static const saint_t lg_table[256]= {
|
||||
-1,0,1,1,2,2,2,2,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,
|
||||
5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,
|
||||
6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,
|
||||
6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,
|
||||
7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,
|
||||
7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,
|
||||
7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,
|
||||
7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7
|
||||
};
|
||||
|
||||
static INLINE
|
||||
saint_t
|
||||
tr_ilg(saidx_t n) {
|
||||
#if defined(BUILD_DIVSUFSORT64)
|
||||
return (n >> 32) ?
|
||||
((n >> 48) ?
|
||||
((n >> 56) ?
|
||||
56 + lg_table[(n >> 56) & 0xff] :
|
||||
48 + lg_table[(n >> 48) & 0xff]) :
|
||||
((n >> 40) ?
|
||||
40 + lg_table[(n >> 40) & 0xff] :
|
||||
32 + lg_table[(n >> 32) & 0xff])) :
|
||||
((n & 0xffff0000) ?
|
||||
((n & 0xff000000) ?
|
||||
24 + lg_table[(n >> 24) & 0xff] :
|
||||
16 + lg_table[(n >> 16) & 0xff]) :
|
||||
((n & 0x0000ff00) ?
|
||||
8 + lg_table[(n >> 8) & 0xff] :
|
||||
0 + lg_table[(n >> 0) & 0xff]));
|
||||
#else
|
||||
return (n & 0xffff0000) ?
|
||||
((n & 0xff000000) ?
|
||||
24 + lg_table[(n >> 24) & 0xff] :
|
||||
16 + lg_table[(n >> 16) & 0xff]) :
|
||||
((n & 0x0000ff00) ?
|
||||
8 + lg_table[(n >> 8) & 0xff] :
|
||||
0 + lg_table[(n >> 0) & 0xff]);
|
||||
#endif
|
||||
}
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
/* Simple insertionsort for small size groups. */
|
||||
static
|
||||
void
|
||||
tr_insertionsort(const saidx_t *ISAd, saidx_t *first, saidx_t *last) {
|
||||
saidx_t *a, *b;
|
||||
saidx_t t, r;
|
||||
|
||||
for(a = first + 1; a < last; ++a) {
|
||||
for(t = *a, b = a - 1; 0 > (r = ISAd[t] - ISAd[*b]);) {
|
||||
do { *(b + 1) = *b; } while((first <= --b) && (*b < 0));
|
||||
if(b < first) { break; }
|
||||
}
|
||||
if(r == 0) { *b = ~*b; }
|
||||
*(b + 1) = t;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
static INLINE
|
||||
void
|
||||
tr_fixdown(const saidx_t *ISAd, saidx_t *SA, saidx_t i, saidx_t size) {
|
||||
saidx_t j, k;
|
||||
saidx_t v;
|
||||
saidx_t c, d, e;
|
||||
|
||||
for(v = SA[i], c = ISAd[v]; (j = 2 * i + 1) < size; SA[i] = SA[k], i = k) {
|
||||
d = ISAd[SA[k = j++]];
|
||||
if(d < (e = ISAd[SA[j]])) { k = j; d = e; }
|
||||
if(d <= c) { break; }
|
||||
}
|
||||
SA[i] = v;
|
||||
}
|
||||
|
||||
/* Simple top-down heapsort. */
|
||||
static
|
||||
void
|
||||
tr_heapsort(const saidx_t *ISAd, saidx_t *SA, saidx_t size) {
|
||||
saidx_t i, m;
|
||||
saidx_t t;
|
||||
|
||||
m = size;
|
||||
if((size % 2) == 0) {
|
||||
m--;
|
||||
if(ISAd[SA[m / 2]] < ISAd[SA[m]]) { SWAP(SA[m], SA[m / 2]); }
|
||||
}
|
||||
|
||||
for(i = m / 2 - 1; 0 <= i; --i) { tr_fixdown(ISAd, SA, i, m); }
|
||||
if((size % 2) == 0) { SWAP(SA[0], SA[m]); tr_fixdown(ISAd, SA, 0, m); }
|
||||
for(i = m - 1; 0 < i; --i) {
|
||||
t = SA[0], SA[0] = SA[i];
|
||||
tr_fixdown(ISAd, SA, 0, i);
|
||||
SA[i] = t;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
/* Returns the median of three elements. */
|
||||
static INLINE
|
||||
saidx_t *
|
||||
tr_median3(const saidx_t *ISAd, saidx_t *v1, saidx_t *v2, saidx_t *v3) {
|
||||
saidx_t *t;
|
||||
if(ISAd[*v1] > ISAd[*v2]) { SWAP(v1, v2); }
|
||||
if(ISAd[*v2] > ISAd[*v3]) {
|
||||
if(ISAd[*v1] > ISAd[*v3]) { return v1; }
|
||||
else { return v3; }
|
||||
}
|
||||
return v2;
|
||||
}
|
||||
|
||||
/* Returns the median of five elements. */
|
||||
static INLINE
|
||||
saidx_t *
|
||||
tr_median5(const saidx_t *ISAd,
|
||||
saidx_t *v1, saidx_t *v2, saidx_t *v3, saidx_t *v4, saidx_t *v5) {
|
||||
saidx_t *t;
|
||||
if(ISAd[*v2] > ISAd[*v3]) { SWAP(v2, v3); }
|
||||
if(ISAd[*v4] > ISAd[*v5]) { SWAP(v4, v5); }
|
||||
if(ISAd[*v2] > ISAd[*v4]) { SWAP(v2, v4); SWAP(v3, v5); }
|
||||
if(ISAd[*v1] > ISAd[*v3]) { SWAP(v1, v3); }
|
||||
if(ISAd[*v1] > ISAd[*v4]) { SWAP(v1, v4); SWAP(v3, v5); }
|
||||
if(ISAd[*v3] > ISAd[*v4]) { return v4; }
|
||||
return v3;
|
||||
}
|
||||
|
||||
/* Returns the pivot element. */
|
||||
static INLINE
|
||||
saidx_t *
|
||||
tr_pivot(const saidx_t *ISAd, saidx_t *first, saidx_t *last) {
|
||||
saidx_t *middle;
|
||||
saidx_t t;
|
||||
|
||||
t = last - first;
|
||||
middle = first + t / 2;
|
||||
|
||||
if(t <= 512) {
|
||||
if(t <= 32) {
|
||||
return tr_median3(ISAd, first, middle, last - 1);
|
||||
} else {
|
||||
t >>= 2;
|
||||
return tr_median5(ISAd, first, first + t, middle, last - 1 - t, last - 1);
|
||||
}
|
||||
}
|
||||
t >>= 3;
|
||||
first = tr_median3(ISAd, first, first + t, first + (t << 1));
|
||||
middle = tr_median3(ISAd, middle - t, middle, middle + t);
|
||||
last = tr_median3(ISAd, last - 1 - (t << 1), last - 1 - t, last - 1);
|
||||
return tr_median3(ISAd, first, middle, last);
|
||||
}
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
typedef struct _trbudget_t trbudget_t;
|
||||
struct _trbudget_t {
|
||||
saidx_t chance;
|
||||
saidx_t remain;
|
||||
saidx_t incval;
|
||||
saidx_t count;
|
||||
};
|
||||
|
||||
static INLINE
|
||||
void
|
||||
trbudget_init(trbudget_t *budget, saidx_t chance, saidx_t incval) {
|
||||
budget->chance = chance;
|
||||
budget->remain = budget->incval = incval;
|
||||
}
|
||||
|
||||
static INLINE
|
||||
saint_t
|
||||
trbudget_check(trbudget_t *budget, saidx_t size) {
|
||||
if(size <= budget->remain) { budget->remain -= size; return 1; }
|
||||
if(budget->chance == 0) { budget->count += size; return 0; }
|
||||
budget->remain += budget->incval - size;
|
||||
budget->chance -= 1;
|
||||
return 1;
|
||||
}
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
static INLINE
|
||||
void
|
||||
tr_partition(const saidx_t *ISAd,
|
||||
saidx_t *first, saidx_t *middle, saidx_t *last,
|
||||
saidx_t **pa, saidx_t **pb, saidx_t v) {
|
||||
saidx_t *a, *b, *c, *d, *e, *f;
|
||||
saidx_t t, s;
|
||||
saidx_t x = 0;
|
||||
|
||||
for(b = middle - 1; (++b < last) && ((x = ISAd[*b]) == v);) { }
|
||||
if(((a = b) < last) && (x < v)) {
|
||||
for(; (++b < last) && ((x = ISAd[*b]) <= v);) {
|
||||
if(x == v) { SWAP(*b, *a); ++a; }
|
||||
}
|
||||
}
|
||||
for(c = last; (b < --c) && ((x = ISAd[*c]) == v);) { }
|
||||
if((b < (d = c)) && (x > v)) {
|
||||
for(; (b < --c) && ((x = ISAd[*c]) >= v);) {
|
||||
if(x == v) { SWAP(*c, *d); --d; }
|
||||
}
|
||||
}
|
||||
for(; b < c;) {
|
||||
SWAP(*b, *c);
|
||||
for(; (++b < c) && ((x = ISAd[*b]) <= v);) {
|
||||
if(x == v) { SWAP(*b, *a); ++a; }
|
||||
}
|
||||
for(; (b < --c) && ((x = ISAd[*c]) >= v);) {
|
||||
if(x == v) { SWAP(*c, *d); --d; }
|
||||
}
|
||||
}
|
||||
|
||||
if(a <= d) {
|
||||
c = b - 1;
|
||||
if((s = a - first) > (t = b - a)) { s = t; }
|
||||
for(e = first, f = b - s; 0 < s; --s, ++e, ++f) { SWAP(*e, *f); }
|
||||
if((s = d - c) > (t = last - d - 1)) { s = t; }
|
||||
for(e = b, f = last - s; 0 < s; --s, ++e, ++f) { SWAP(*e, *f); }
|
||||
first += (b - a), last -= (d - c);
|
||||
}
|
||||
*pa = first, *pb = last;
|
||||
}
|
||||
|
||||
static
|
||||
void
|
||||
tr_copy(saidx_t *ISA, const saidx_t *SA,
|
||||
saidx_t *first, saidx_t *a, saidx_t *b, saidx_t *last,
|
||||
saidx_t depth) {
|
||||
/* sort suffixes of middle partition
|
||||
by using sorted order of suffixes of left and right partition. */
|
||||
saidx_t *c, *d, *e;
|
||||
saidx_t s, v;
|
||||
|
||||
v = b - SA - 1;
|
||||
for(c = first, d = a - 1; c <= d; ++c) {
|
||||
if((0 <= (s = *c - depth)) && (ISA[s] == v)) {
|
||||
*++d = s;
|
||||
ISA[s] = d - SA;
|
||||
}
|
||||
}
|
||||
for(c = last - 1, e = d + 1, d = b; e < d; --c) {
|
||||
if((0 <= (s = *c - depth)) && (ISA[s] == v)) {
|
||||
*--d = s;
|
||||
ISA[s] = d - SA;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static
|
||||
void
|
||||
tr_partialcopy(saidx_t *ISA, const saidx_t *SA,
|
||||
saidx_t *first, saidx_t *a, saidx_t *b, saidx_t *last,
|
||||
saidx_t depth) {
|
||||
saidx_t *c, *d, *e;
|
||||
saidx_t s, v;
|
||||
saidx_t rank, lastrank, newrank = -1;
|
||||
|
||||
v = b - SA - 1;
|
||||
lastrank = -1;
|
||||
for(c = first, d = a - 1; c <= d; ++c) {
|
||||
if((0 <= (s = *c - depth)) && (ISA[s] == v)) {
|
||||
*++d = s;
|
||||
rank = ISA[s + depth];
|
||||
if(lastrank != rank) { lastrank = rank; newrank = d - SA; }
|
||||
ISA[s] = newrank;
|
||||
}
|
||||
}
|
||||
|
||||
lastrank = -1;
|
||||
for(e = d; first <= e; --e) {
|
||||
rank = ISA[*e];
|
||||
if(lastrank != rank) { lastrank = rank; newrank = e - SA; }
|
||||
if(newrank != rank) { ISA[*e] = newrank; }
|
||||
}
|
||||
|
||||
lastrank = -1;
|
||||
for(c = last - 1, e = d + 1, d = b; e < d; --c) {
|
||||
if((0 <= (s = *c - depth)) && (ISA[s] == v)) {
|
||||
*--d = s;
|
||||
rank = ISA[s + depth];
|
||||
if(lastrank != rank) { lastrank = rank; newrank = d - SA; }
|
||||
ISA[s] = newrank;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static
|
||||
void
|
||||
tr_introsort(saidx_t *ISA, const saidx_t *ISAd,
|
||||
saidx_t *SA, saidx_t *first, saidx_t *last,
|
||||
trbudget_t *budget) {
|
||||
#define STACK_SIZE TR_STACKSIZE
|
||||
struct { const saidx_t *a; saidx_t *b, *c; saint_t d, e; }stack[STACK_SIZE];
|
||||
saidx_t *a, *b, *c;
|
||||
saidx_t t;
|
||||
saidx_t v, x = 0;
|
||||
saidx_t incr = ISAd - ISA;
|
||||
saint_t limit, next;
|
||||
saint_t ssize, trlink = -1;
|
||||
|
||||
for(ssize = 0, limit = tr_ilg(last - first);;) {
|
||||
|
||||
if(limit < 0) {
|
||||
if(limit == -1) {
|
||||
/* tandem repeat partition */
|
||||
tr_partition(ISAd - incr, first, first, last, &a, &b, last - SA - 1);
|
||||
|
||||
/* update ranks */
|
||||
if(a < last) {
|
||||
for(c = first, v = a - SA - 1; c < a; ++c) { ISA[*c] = v; }
|
||||
}
|
||||
if(b < last) {
|
||||
for(c = a, v = b - SA - 1; c < b; ++c) { ISA[*c] = v; }
|
||||
}
|
||||
|
||||
/* push */
|
||||
if(1 < (b - a)) {
|
||||
STACK_PUSH5(NULL, a, b, 0, 0);
|
||||
STACK_PUSH5(ISAd - incr, first, last, -2, trlink);
|
||||
trlink = ssize - 2;
|
||||
}
|
||||
if((a - first) <= (last - b)) {
|
||||
if(1 < (a - first)) {
|
||||
STACK_PUSH5(ISAd, b, last, tr_ilg(last - b), trlink);
|
||||
last = a, limit = tr_ilg(a - first);
|
||||
} else if(1 < (last - b)) {
|
||||
first = b, limit = tr_ilg(last - b);
|
||||
} else {
|
||||
STACK_POP5(ISAd, first, last, limit, trlink);
|
||||
}
|
||||
} else {
|
||||
if(1 < (last - b)) {
|
||||
STACK_PUSH5(ISAd, first, a, tr_ilg(a - first), trlink);
|
||||
first = b, limit = tr_ilg(last - b);
|
||||
} else if(1 < (a - first)) {
|
||||
last = a, limit = tr_ilg(a - first);
|
||||
} else {
|
||||
STACK_POP5(ISAd, first, last, limit, trlink);
|
||||
}
|
||||
}
|
||||
} else if(limit == -2) {
|
||||
/* tandem repeat copy */
|
||||
a = stack[--ssize].b, b = stack[ssize].c;
|
||||
if(stack[ssize].d == 0) {
|
||||
tr_copy(ISA, SA, first, a, b, last, ISAd - ISA);
|
||||
} else {
|
||||
if(0 <= trlink) { stack[trlink].d = -1; }
|
||||
tr_partialcopy(ISA, SA, first, a, b, last, ISAd - ISA);
|
||||
}
|
||||
STACK_POP5(ISAd, first, last, limit, trlink);
|
||||
} else {
|
||||
/* sorted partition */
|
||||
if(0 <= *first) {
|
||||
a = first;
|
||||
do { ISA[*a] = a - SA; } while((++a < last) && (0 <= *a));
|
||||
first = a;
|
||||
}
|
||||
if(first < last) {
|
||||
a = first; do { *a = ~*a; } while(*++a < 0);
|
||||
next = (ISA[*a] != ISAd[*a]) ? tr_ilg(a - first + 1) : -1;
|
||||
if(++a < last) { for(b = first, v = a - SA - 1; b < a; ++b) { ISA[*b] = v; } }
|
||||
|
||||
/* push */
|
||||
if(trbudget_check(budget, a - first)) {
|
||||
if((a - first) <= (last - a)) {
|
||||
STACK_PUSH5(ISAd, a, last, -3, trlink);
|
||||
ISAd += incr, last = a, limit = next;
|
||||
} else {
|
||||
if(1 < (last - a)) {
|
||||
STACK_PUSH5(ISAd + incr, first, a, next, trlink);
|
||||
first = a, limit = -3;
|
||||
} else {
|
||||
ISAd += incr, last = a, limit = next;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if(0 <= trlink) { stack[trlink].d = -1; }
|
||||
if(1 < (last - a)) {
|
||||
first = a, limit = -3;
|
||||
} else {
|
||||
STACK_POP5(ISAd, first, last, limit, trlink);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
STACK_POP5(ISAd, first, last, limit, trlink);
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if((last - first) <= TR_INSERTIONSORT_THRESHOLD) {
|
||||
tr_insertionsort(ISAd, first, last);
|
||||
limit = -3;
|
||||
continue;
|
||||
}
|
||||
|
||||
if(limit-- == 0) {
|
||||
tr_heapsort(ISAd, first, last - first);
|
||||
for(a = last - 1; first < a; a = b) {
|
||||
for(x = ISAd[*a], b = a - 1; (first <= b) && (ISAd[*b] == x); --b) { *b = ~*b; }
|
||||
}
|
||||
limit = -3;
|
||||
continue;
|
||||
}
|
||||
|
||||
/* choose pivot */
|
||||
a = tr_pivot(ISAd, first, last);
|
||||
SWAP(*first, *a);
|
||||
v = ISAd[*first];
|
||||
|
||||
/* partition */
|
||||
tr_partition(ISAd, first, first + 1, last, &a, &b, v);
|
||||
if((last - first) != (b - a)) {
|
||||
next = (ISA[*a] != v) ? tr_ilg(b - a) : -1;
|
||||
|
||||
/* update ranks */
|
||||
for(c = first, v = a - SA - 1; c < a; ++c) { ISA[*c] = v; }
|
||||
if(b < last) { for(c = a, v = b - SA - 1; c < b; ++c) { ISA[*c] = v; } }
|
||||
|
||||
/* push */
|
||||
if((1 < (b - a)) && (trbudget_check(budget, b - a))) {
|
||||
if((a - first) <= (last - b)) {
|
||||
if((last - b) <= (b - a)) {
|
||||
if(1 < (a - first)) {
|
||||
STACK_PUSH5(ISAd + incr, a, b, next, trlink);
|
||||
STACK_PUSH5(ISAd, b, last, limit, trlink);
|
||||
last = a;
|
||||
} else if(1 < (last - b)) {
|
||||
STACK_PUSH5(ISAd + incr, a, b, next, trlink);
|
||||
first = b;
|
||||
} else {
|
||||
ISAd += incr, first = a, last = b, limit = next;
|
||||
}
|
||||
} else if((a - first) <= (b - a)) {
|
||||
if(1 < (a - first)) {
|
||||
STACK_PUSH5(ISAd, b, last, limit, trlink);
|
||||
STACK_PUSH5(ISAd + incr, a, b, next, trlink);
|
||||
last = a;
|
||||
} else {
|
||||
STACK_PUSH5(ISAd, b, last, limit, trlink);
|
||||
ISAd += incr, first = a, last = b, limit = next;
|
||||
}
|
||||
} else {
|
||||
STACK_PUSH5(ISAd, b, last, limit, trlink);
|
||||
STACK_PUSH5(ISAd, first, a, limit, trlink);
|
||||
ISAd += incr, first = a, last = b, limit = next;
|
||||
}
|
||||
} else {
|
||||
if((a - first) <= (b - a)) {
|
||||
if(1 < (last - b)) {
|
||||
STACK_PUSH5(ISAd + incr, a, b, next, trlink);
|
||||
STACK_PUSH5(ISAd, first, a, limit, trlink);
|
||||
first = b;
|
||||
} else if(1 < (a - first)) {
|
||||
STACK_PUSH5(ISAd + incr, a, b, next, trlink);
|
||||
last = a;
|
||||
} else {
|
||||
ISAd += incr, first = a, last = b, limit = next;
|
||||
}
|
||||
} else if((last - b) <= (b - a)) {
|
||||
if(1 < (last - b)) {
|
||||
STACK_PUSH5(ISAd, first, a, limit, trlink);
|
||||
STACK_PUSH5(ISAd + incr, a, b, next, trlink);
|
||||
first = b;
|
||||
} else {
|
||||
STACK_PUSH5(ISAd, first, a, limit, trlink);
|
||||
ISAd += incr, first = a, last = b, limit = next;
|
||||
}
|
||||
} else {
|
||||
STACK_PUSH5(ISAd, first, a, limit, trlink);
|
||||
STACK_PUSH5(ISAd, b, last, limit, trlink);
|
||||
ISAd += incr, first = a, last = b, limit = next;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if((1 < (b - a)) && (0 <= trlink)) { stack[trlink].d = -1; }
|
||||
if((a - first) <= (last - b)) {
|
||||
if(1 < (a - first)) {
|
||||
STACK_PUSH5(ISAd, b, last, limit, trlink);
|
||||
last = a;
|
||||
} else if(1 < (last - b)) {
|
||||
first = b;
|
||||
} else {
|
||||
STACK_POP5(ISAd, first, last, limit, trlink);
|
||||
}
|
||||
} else {
|
||||
if(1 < (last - b)) {
|
||||
STACK_PUSH5(ISAd, first, a, limit, trlink);
|
||||
first = b;
|
||||
} else if(1 < (a - first)) {
|
||||
last = a;
|
||||
} else {
|
||||
STACK_POP5(ISAd, first, last, limit, trlink);
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if(trbudget_check(budget, last - first)) {
|
||||
limit = tr_ilg(last - first), ISAd += incr;
|
||||
} else {
|
||||
if(0 <= trlink) { stack[trlink].d = -1; }
|
||||
STACK_POP5(ISAd, first, last, limit, trlink);
|
||||
}
|
||||
}
|
||||
}
|
||||
#undef STACK_SIZE
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
/*- Function -*/
|
||||
|
||||
/* Tandem repeat sort */
|
||||
void
|
||||
trsort(saidx_t *ISA, saidx_t *SA, saidx_t n, saidx_t depth) {
|
||||
saidx_t *ISAd;
|
||||
saidx_t *first, *last;
|
||||
trbudget_t budget;
|
||||
saidx_t t, skip, unsorted;
|
||||
|
||||
trbudget_init(&budget, tr_ilg(n) * 2 / 3, n);
|
||||
/* trbudget_init(&budget, tr_ilg(n) * 3 / 4, n); */
|
||||
for(ISAd = ISA + depth; -n < *SA; ISAd += ISAd - ISA) {
|
||||
first = SA;
|
||||
skip = 0;
|
||||
unsorted = 0;
|
||||
do {
|
||||
if((t = *first) < 0) { first -= t; skip += t; }
|
||||
else {
|
||||
if(skip != 0) { *(first + skip) = skip; skip = 0; }
|
||||
last = SA + ISA[t] + 1;
|
||||
if(1 < (last - first)) {
|
||||
budget.count = 0;
|
||||
tr_introsort(ISA, ISAd, SA, first, last, &budget);
|
||||
if(budget.count != 0) { unsorted += budget.count; }
|
||||
else { skip = first - last; }
|
||||
} else if((last - first) == 1) {
|
||||
skip = -1;
|
||||
}
|
||||
first = last;
|
||||
}
|
||||
} while(first < (SA + n));
|
||||
if(skip != 0) { *(first + skip) = skip; }
|
||||
if(unsorted == 0) { break; }
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,383 @@
|
|||
/*
|
||||
* utils.c for libdivsufsort
|
||||
* Copyright (c) 2003-2008 Yuta Mori All Rights Reserved.
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person
|
||||
* obtaining a copy of this software and associated documentation
|
||||
* files (the "Software"), to deal in the Software without
|
||||
* restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the
|
||||
* Software is furnished to do so, subject to the following
|
||||
* conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be
|
||||
* included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#include "divsufsort_private.h"
|
||||
|
||||
|
||||
/*- Private Function -*/
|
||||
|
||||
#if 0
|
||||
/* Binary search for inverse bwt. */
|
||||
static
|
||||
saidx_t
|
||||
binarysearch_lower(const saidx_t *A, saidx_t size, saidx_t value) {
|
||||
saidx_t half, i;
|
||||
for(i = 0, half = size >> 1;
|
||||
0 < size;
|
||||
size = half, half >>= 1) {
|
||||
if(A[i + half] < value) {
|
||||
i += half + 1;
|
||||
half -= (size & 1) ^ 1;
|
||||
}
|
||||
}
|
||||
return i;
|
||||
}
|
||||
|
||||
|
||||
/*- Functions -*/
|
||||
|
||||
/* Burrows-Wheeler transform. */
|
||||
saint_t
|
||||
bw_transform(const sauchar_t *T, sauchar_t *U, saidx_t *SA,
|
||||
saidx_t n, saidx_t *idx) {
|
||||
saidx_t *A, i, j, p, t;
|
||||
saint_t c;
|
||||
|
||||
/* Check arguments. */
|
||||
if((T == NULL) || (U == NULL) || (n < 0) || (idx == NULL)) { return -1; }
|
||||
if(n <= 1) {
|
||||
if(n == 1) { U[0] = T[0]; }
|
||||
*idx = n;
|
||||
return 0;
|
||||
}
|
||||
|
||||
if((A = SA) == NULL) {
|
||||
i = divbwt(T, U, NULL, n);
|
||||
if(0 <= i) { *idx = i; i = 0; }
|
||||
return (saint_t)i;
|
||||
}
|
||||
|
||||
/* BW transform. */
|
||||
if(T == U) {
|
||||
t = n;
|
||||
for(i = 0, j = 0; i < n; ++i) {
|
||||
p = t - 1;
|
||||
t = A[i];
|
||||
if(0 <= p) {
|
||||
c = T[j];
|
||||
U[j] = (j <= p) ? T[p] : (sauchar_t)A[p];
|
||||
A[j] = c;
|
||||
j++;
|
||||
} else {
|
||||
*idx = i;
|
||||
}
|
||||
}
|
||||
p = t - 1;
|
||||
if(0 <= p) {
|
||||
c = T[j];
|
||||
U[j] = (j <= p) ? T[p] : (sauchar_t)A[p];
|
||||
A[j] = c;
|
||||
} else {
|
||||
*idx = i;
|
||||
}
|
||||
} else {
|
||||
U[0] = T[n - 1];
|
||||
for(i = 0; A[i] != 0; ++i) { U[i + 1] = T[A[i] - 1]; }
|
||||
*idx = i + 1;
|
||||
for(++i; i < n; ++i) { U[i] = T[A[i] - 1]; }
|
||||
}
|
||||
|
||||
if(SA == NULL) {
|
||||
/* Deallocate memory. */
|
||||
free(A);
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Inverse Burrows-Wheeler transform. */
|
||||
saint_t
|
||||
inverse_bw_transform(const sauchar_t *T, sauchar_t *U, saidx_t *A,
|
||||
saidx_t n, saidx_t idx) {
|
||||
saidx_t C[ALPHABET_SIZE];
|
||||
sauchar_t D[ALPHABET_SIZE];
|
||||
saidx_t *B;
|
||||
saidx_t i, p;
|
||||
saint_t c, d;
|
||||
|
||||
/* Check arguments. */
|
||||
if((T == NULL) || (U == NULL) || (n < 0) || (idx < 0) ||
|
||||
(n < idx) || ((0 < n) && (idx == 0))) {
|
||||
return -1;
|
||||
}
|
||||
if(n <= 1) { return 0; }
|
||||
|
||||
if((B = A) == NULL) {
|
||||
/* Allocate n*sizeof(saidx_t) bytes of memory. */
|
||||
if((B = (saidx_t *)malloc((size_t)n * sizeof(saidx_t))) == NULL) { return -2; }
|
||||
}
|
||||
|
||||
/* Inverse BW transform. */
|
||||
for(c = 0; c < ALPHABET_SIZE; ++c) { C[c] = 0; }
|
||||
for(i = 0; i < n; ++i) { ++C[T[i]]; }
|
||||
for(c = 0, d = 0, i = 0; c < ALPHABET_SIZE; ++c) {
|
||||
p = C[c];
|
||||
if(0 < p) {
|
||||
C[c] = i;
|
||||
D[d++] = (sauchar_t)c;
|
||||
i += p;
|
||||
}
|
||||
}
|
||||
for(i = 0; i < idx; ++i) { B[C[T[i]]++] = i; }
|
||||
for( ; i < n; ++i) { B[C[T[i]]++] = i + 1; }
|
||||
for(c = 0; c < d; ++c) { C[c] = C[D[c]]; }
|
||||
for(i = 0, p = idx; i < n; ++i) {
|
||||
U[i] = D[binarysearch_lower(C, d, p)];
|
||||
p = B[p - 1];
|
||||
}
|
||||
|
||||
if(A == NULL) {
|
||||
/* Deallocate memory. */
|
||||
free(B);
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Checks the suffix array SA of the string T. */
|
||||
saint_t
|
||||
sufcheck(const sauchar_t *T, const saidx_t *SA,
|
||||
saidx_t n, saint_t verbose) {
|
||||
saidx_t C[ALPHABET_SIZE];
|
||||
saidx_t i, p, q, t;
|
||||
saint_t c;
|
||||
|
||||
if(verbose) { fprintf(stderr, "sufcheck: "); }
|
||||
|
||||
/* Check arguments. */
|
||||
if((T == NULL) || (SA == NULL) || (n < 0)) {
|
||||
if(verbose) { fprintf(stderr, "Invalid arguments.\n"); }
|
||||
return -1;
|
||||
}
|
||||
if(n == 0) {
|
||||
if(verbose) { fprintf(stderr, "Done.\n"); }
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* check range: [0..n-1] */
|
||||
for(i = 0; i < n; ++i) {
|
||||
if((SA[i] < 0) || (n <= SA[i])) {
|
||||
if(verbose) {
|
||||
fprintf(stderr, "Out of the range [0,%" PRIdSAIDX_T "].\n"
|
||||
" SA[%" PRIdSAIDX_T "]=%" PRIdSAIDX_T "\n",
|
||||
n - 1, i, SA[i]);
|
||||
}
|
||||
return -2;
|
||||
}
|
||||
}
|
||||
|
||||
/* check first characters. */
|
||||
for(i = 1; i < n; ++i) {
|
||||
if(T[SA[i - 1]] > T[SA[i]]) {
|
||||
if(verbose) {
|
||||
fprintf(stderr, "Suffixes in wrong order.\n"
|
||||
" T[SA[%" PRIdSAIDX_T "]=%" PRIdSAIDX_T "]=%d"
|
||||
" > T[SA[%" PRIdSAIDX_T "]=%" PRIdSAIDX_T "]=%d\n",
|
||||
i - 1, SA[i - 1], T[SA[i - 1]], i, SA[i], T[SA[i]]);
|
||||
}
|
||||
return -3;
|
||||
}
|
||||
}
|
||||
|
||||
/* check suffixes. */
|
||||
for(i = 0; i < ALPHABET_SIZE; ++i) { C[i] = 0; }
|
||||
for(i = 0; i < n; ++i) { ++C[T[i]]; }
|
||||
for(i = 0, p = 0; i < ALPHABET_SIZE; ++i) {
|
||||
t = C[i];
|
||||
C[i] = p;
|
||||
p += t;
|
||||
}
|
||||
|
||||
q = C[T[n - 1]];
|
||||
C[T[n - 1]] += 1;
|
||||
for(i = 0; i < n; ++i) {
|
||||
p = SA[i];
|
||||
if(0 < p) {
|
||||
c = T[--p];
|
||||
t = C[c];
|
||||
} else {
|
||||
c = T[p = n - 1];
|
||||
t = q;
|
||||
}
|
||||
if((t < 0) || (p != SA[t])) {
|
||||
if(verbose) {
|
||||
fprintf(stderr, "Suffix in wrong position.\n"
|
||||
" SA[%" PRIdSAIDX_T "]=%" PRIdSAIDX_T " or\n"
|
||||
" SA[%" PRIdSAIDX_T "]=%" PRIdSAIDX_T "\n",
|
||||
t, (0 <= t) ? SA[t] : -1, i, SA[i]);
|
||||
}
|
||||
return -4;
|
||||
}
|
||||
if(t != q) {
|
||||
++C[c];
|
||||
if((n <= C[c]) || (T[SA[C[c]]] != c)) { C[c] = -1; }
|
||||
}
|
||||
}
|
||||
|
||||
if(1 <= verbose) { fprintf(stderr, "Done.\n"); }
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
static
|
||||
int
|
||||
_compare(const sauchar_t *T, saidx_t Tsize,
|
||||
const sauchar_t *P, saidx_t Psize,
|
||||
saidx_t suf, saidx_t *match) {
|
||||
saidx_t i, j;
|
||||
saint_t r;
|
||||
for(i = suf + *match, j = *match, r = 0;
|
||||
(i < Tsize) && (j < Psize) && ((r = T[i] - P[j]) == 0); ++i, ++j) { }
|
||||
*match = j;
|
||||
return (r == 0) ? -(j != Psize) : r;
|
||||
}
|
||||
|
||||
/* Search for the pattern P in the string T. */
|
||||
saidx_t
|
||||
sa_search(const sauchar_t *T, saidx_t Tsize,
|
||||
const sauchar_t *P, saidx_t Psize,
|
||||
const saidx_t *SA, saidx_t SAsize,
|
||||
saidx_t *idx) {
|
||||
saidx_t size, lsize, rsize, half;
|
||||
saidx_t match, lmatch, rmatch;
|
||||
saidx_t llmatch, lrmatch, rlmatch, rrmatch;
|
||||
saidx_t i, j, k;
|
||||
saint_t r;
|
||||
|
||||
if(idx != NULL) { *idx = -1; }
|
||||
if((T == NULL) || (P == NULL) || (SA == NULL) ||
|
||||
(Tsize < 0) || (Psize < 0) || (SAsize < 0)) { return -1; }
|
||||
if((Tsize == 0) || (SAsize == 0)) { return 0; }
|
||||
if(Psize == 0) { if(idx != NULL) { *idx = 0; } return SAsize; }
|
||||
|
||||
for(i = j = k = 0, lmatch = rmatch = 0, size = SAsize, half = size >> 1;
|
||||
0 < size;
|
||||
size = half, half >>= 1) {
|
||||
match = MIN(lmatch, rmatch);
|
||||
r = _compare(T, Tsize, P, Psize, SA[i + half], &match);
|
||||
if(r < 0) {
|
||||
i += half + 1;
|
||||
half -= (size & 1) ^ 1;
|
||||
lmatch = match;
|
||||
} else if(r > 0) {
|
||||
rmatch = match;
|
||||
} else {
|
||||
lsize = half, j = i, rsize = size - half - 1, k = i + half + 1;
|
||||
|
||||
/* left part */
|
||||
for(llmatch = lmatch, lrmatch = match, half = lsize >> 1;
|
||||
0 < lsize;
|
||||
lsize = half, half >>= 1) {
|
||||
lmatch = MIN(llmatch, lrmatch);
|
||||
r = _compare(T, Tsize, P, Psize, SA[j + half], &lmatch);
|
||||
if(r < 0) {
|
||||
j += half + 1;
|
||||
half -= (lsize & 1) ^ 1;
|
||||
llmatch = lmatch;
|
||||
} else {
|
||||
lrmatch = lmatch;
|
||||
}
|
||||
}
|
||||
|
||||
/* right part */
|
||||
for(rlmatch = match, rrmatch = rmatch, half = rsize >> 1;
|
||||
0 < rsize;
|
||||
rsize = half, half >>= 1) {
|
||||
rmatch = MIN(rlmatch, rrmatch);
|
||||
r = _compare(T, Tsize, P, Psize, SA[k + half], &rmatch);
|
||||
if(r <= 0) {
|
||||
k += half + 1;
|
||||
half -= (rsize & 1) ^ 1;
|
||||
rlmatch = rmatch;
|
||||
} else {
|
||||
rrmatch = rmatch;
|
||||
}
|
||||
}
|
||||
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if(idx != NULL) { *idx = (0 < (k - j)) ? j : i; }
|
||||
return k - j;
|
||||
}
|
||||
|
||||
/* Search for the character c in the string T. */
|
||||
saidx_t
|
||||
sa_simplesearch(const sauchar_t *T, saidx_t Tsize,
|
||||
const saidx_t *SA, saidx_t SAsize,
|
||||
saint_t c, saidx_t *idx) {
|
||||
saidx_t size, lsize, rsize, half;
|
||||
saidx_t i, j, k, p;
|
||||
saint_t r;
|
||||
|
||||
if(idx != NULL) { *idx = -1; }
|
||||
if((T == NULL) || (SA == NULL) || (Tsize < 0) || (SAsize < 0)) { return -1; }
|
||||
if((Tsize == 0) || (SAsize == 0)) { return 0; }
|
||||
|
||||
for(i = j = k = 0, size = SAsize, half = size >> 1;
|
||||
0 < size;
|
||||
size = half, half >>= 1) {
|
||||
p = SA[i + half];
|
||||
r = (p < Tsize) ? T[p] - c : -1;
|
||||
if(r < 0) {
|
||||
i += half + 1;
|
||||
half -= (size & 1) ^ 1;
|
||||
} else if(r == 0) {
|
||||
lsize = half, j = i, rsize = size - half - 1, k = i + half + 1;
|
||||
|
||||
/* left part */
|
||||
for(half = lsize >> 1;
|
||||
0 < lsize;
|
||||
lsize = half, half >>= 1) {
|
||||
p = SA[j + half];
|
||||
r = (p < Tsize) ? T[p] - c : -1;
|
||||
if(r < 0) {
|
||||
j += half + 1;
|
||||
half -= (lsize & 1) ^ 1;
|
||||
}
|
||||
}
|
||||
|
||||
/* right part */
|
||||
for(half = rsize >> 1;
|
||||
0 < rsize;
|
||||
rsize = half, half >>= 1) {
|
||||
p = SA[k + half];
|
||||
r = (p < Tsize) ? T[p] - c : -1;
|
||||
if(r <= 0) {
|
||||
k += half + 1;
|
||||
half -= (rsize & 1) ^ 1;
|
||||
}
|
||||
}
|
||||
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if(idx != NULL) { *idx = (0 < (k - j)) ? j : i; }
|
||||
return k - j;
|
||||
}
|
||||
#endif
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
## generate libdivsufsort.pc ##
|
||||
set(W64BIT "")
|
||||
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/libdivsufsort.pc.cmake" "${CMAKE_CURRENT_BINARY_DIR}/libdivsufsort.pc" @ONLY)
|
||||
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/libdivsufsort.pc" DESTINATION ${CMAKE_INSTALL_PKGCONFIGDIR})
|
||||
if(BUILD_DIVSUFSORT64)
|
||||
set(W64BIT "64")
|
||||
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/libdivsufsort.pc.cmake" "${CMAKE_CURRENT_BINARY_DIR}/libdivsufsort64.pc" @ONLY)
|
||||
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/libdivsufsort64.pc" DESTINATION ${CMAKE_INSTALL_PKGCONFIGDIR})
|
||||
endif(BUILD_DIVSUFSORT64)
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
prefix=@CMAKE_INSTALL_PREFIX@
|
||||
exec_prefix=${prefix}
|
||||
libdir=@CMAKE_INSTALL_LIBDIR@
|
||||
includedir=@CMAKE_INSTALL_INCLUDEDIR@
|
||||
|
||||
Name: @PROJECT_NAME@@W64BIT@
|
||||
Description: @PROJECT_DESCRIPTION@
|
||||
Version: @PROJECT_VERSION_FULL@
|
||||
URL: @PROJECT_URL@
|
||||
Libs: -L${libdir} -ldivsufsort@W64BIT@
|
||||
Cflags: -I${includedir}
|
||||
|
|
@ -0,0 +1,660 @@
|
|||
/*
|
||||
* main.c - command line compression utility for the LZSA format
|
||||
*
|
||||
* Copyright (C) 2019 Emmanuel Marty
|
||||
*
|
||||
* This software is provided 'as-is', without any express or implied
|
||||
* warranty. In no event will the authors be held liable for any damages
|
||||
* arising from the use of this software.
|
||||
*
|
||||
* Permission is granted to anyone to use this software for any purpose,
|
||||
* including commercial applications, and to alter it and redistribute it
|
||||
* freely, subject to the following restrictions:
|
||||
*
|
||||
* 1. The origin of this software must not be misrepresented; you must not
|
||||
* claim that you wrote the original software. If you use this software
|
||||
* in a product, an acknowledgment in the product documentation would be
|
||||
* appreciated but is not required.
|
||||
* 2. Altered source versions must be plainly marked as such, and must not be
|
||||
* misrepresented as being the original software.
|
||||
* 3. This notice may not be removed or altered from any source distribution.
|
||||
*/
|
||||
|
||||
#include <stdio.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#ifdef _WIN32
|
||||
#include <sys/timeb.h>
|
||||
#else
|
||||
#include <sys/time.h>
|
||||
#endif
|
||||
#include "format.h"
|
||||
#include "shrink.h"
|
||||
#include "expand.h"
|
||||
|
||||
#define BLOCK_SIZE 65536
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
static long long lzsa_get_time() {
|
||||
long long nTime;
|
||||
|
||||
#ifdef _WIN32
|
||||
struct _timeb tb;
|
||||
_ftime(&tb);
|
||||
|
||||
nTime = ((long long)tb.time * 1000LL + (long long)tb.millitm) * 1000LL;
|
||||
#else
|
||||
struct timeval tm;
|
||||
gettimeofday(&tm, NULL);
|
||||
|
||||
nTime = (long long)tm.tv_sec * 1000000LL + (long long)tm.tv_usec;
|
||||
#endif
|
||||
return nTime;
|
||||
}
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
static int lzsa_compress(const char *pszInFilename, const char *pszOutFilename, int nVerbosity) {
|
||||
FILE *f_in, *f_out;
|
||||
unsigned char *pInData, *pOutData;
|
||||
lsza_compressor compressor;
|
||||
long long nStartTime = 0LL, nEndTime = 0LL;
|
||||
long long nOriginalSize = 0LL, nCompressedSize = 0LL;
|
||||
int nResult;
|
||||
bool bError;
|
||||
|
||||
f_in = fopen(pszInFilename, "rb");
|
||||
if (!f_in) {
|
||||
fprintf(stderr, "error opening '%s' for reading\n", pszInFilename);
|
||||
return 100;
|
||||
}
|
||||
|
||||
f_out = fopen(pszOutFilename, "wb");
|
||||
if (!f_out) {
|
||||
fprintf(stderr, "error opening '%s' for writing\n", pszOutFilename);
|
||||
return 100;
|
||||
}
|
||||
|
||||
pInData = (unsigned char*)malloc(BLOCK_SIZE * 2);
|
||||
if (!pInData) {
|
||||
fclose(f_out);
|
||||
f_out = NULL;
|
||||
|
||||
fclose(f_in);
|
||||
f_in = NULL;
|
||||
|
||||
fprintf(stderr, "out of memory\n");
|
||||
return 100;
|
||||
}
|
||||
memset(pInData, 0, BLOCK_SIZE * 2);
|
||||
|
||||
pOutData = (unsigned char*)malloc(BLOCK_SIZE);
|
||||
if (!pOutData) {
|
||||
free(pInData);
|
||||
pInData = NULL;
|
||||
|
||||
fclose(f_out);
|
||||
f_out = NULL;
|
||||
|
||||
fclose(f_in);
|
||||
f_in = NULL;
|
||||
|
||||
fprintf(stderr, "out of memory\n");
|
||||
return 100;
|
||||
}
|
||||
memset(pInData, 0, BLOCK_SIZE);
|
||||
|
||||
nResult = lzsa_compressor_init(&compressor, BLOCK_SIZE * 2);
|
||||
if (nResult != 0) {
|
||||
free(pOutData);
|
||||
pOutData = NULL;
|
||||
|
||||
free(pInData);
|
||||
pInData = NULL;
|
||||
|
||||
fclose(f_out);
|
||||
f_out = NULL;
|
||||
|
||||
fclose(f_in);
|
||||
f_in = NULL;
|
||||
|
||||
fprintf(stderr, "error initializing compressor\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
unsigned char cHeader[5];
|
||||
|
||||
cHeader[0] = 0x7b; /* Magic number: 0xd70f9e7b */
|
||||
cHeader[1] = 0x9e;
|
||||
cHeader[2] = 0x0f;
|
||||
cHeader[3] = 0xd7;
|
||||
cHeader[4] = 0; /* Format version 1 */
|
||||
|
||||
bError = fwrite(cHeader, 1, 5, f_out) != 5;
|
||||
nCompressedSize += 5LL;
|
||||
|
||||
if (nVerbosity > 0) {
|
||||
nStartTime = lzsa_get_time();
|
||||
}
|
||||
|
||||
int nPreviousBlockSize = 0;
|
||||
|
||||
while (!feof(f_in) && !bError) {
|
||||
int nInDataSize;
|
||||
|
||||
if (nPreviousBlockSize) {
|
||||
memcpy(pInData, pInData + BLOCK_SIZE, nPreviousBlockSize);
|
||||
}
|
||||
|
||||
nInDataSize = (int)fread(pInData + BLOCK_SIZE, 1, BLOCK_SIZE, f_in);
|
||||
if (nInDataSize > 0) {
|
||||
int nOutDataSize;
|
||||
|
||||
nOutDataSize = lzsa_shrink_block(&compressor, pInData + BLOCK_SIZE - nPreviousBlockSize, nPreviousBlockSize, nInDataSize, pOutData, (nInDataSize >= BLOCK_SIZE) ? BLOCK_SIZE : nInDataSize);
|
||||
if (nOutDataSize >= 0) {
|
||||
/* Write compressed block */
|
||||
|
||||
unsigned char cBlockSize[3];
|
||||
|
||||
cBlockSize[0] = nOutDataSize & 0xff;
|
||||
cBlockSize[1] = (nOutDataSize >> 8) & 0xff;
|
||||
cBlockSize[2] = (nOutDataSize >> 16) & 0xff;
|
||||
|
||||
if (fwrite(cBlockSize, 1, 3, f_out) != (size_t)3) {
|
||||
bError = true;
|
||||
}
|
||||
else {
|
||||
if (fwrite(pOutData, 1, (size_t)nOutDataSize, f_out) != (size_t)nOutDataSize) {
|
||||
bError = true;
|
||||
}
|
||||
else {
|
||||
nOriginalSize += (long long)nInDataSize;
|
||||
nCompressedSize += 3LL + (long long)nOutDataSize;
|
||||
}
|
||||
}
|
||||
}
|
||||
else {
|
||||
/* Write uncompressible, literal block */
|
||||
|
||||
unsigned char cBlockSize[3];
|
||||
|
||||
cBlockSize[0] = nInDataSize & 0xff;
|
||||
cBlockSize[1] = (nInDataSize >> 8) & 0xff;
|
||||
cBlockSize[2] = ((nInDataSize >> 16) & 0x7f) | 0x80; /* Uncompressed block */
|
||||
|
||||
if (fwrite(cBlockSize, 1, 3, f_out) != (size_t)3) {
|
||||
bError = true;
|
||||
}
|
||||
else {
|
||||
if (fwrite(pInData + BLOCK_SIZE, 1, (size_t)nInDataSize, f_out) != (size_t)nInDataSize) {
|
||||
bError = true;
|
||||
}
|
||||
else {
|
||||
nOriginalSize += (long long)nInDataSize;
|
||||
nCompressedSize += 3LL + (long long)nInDataSize;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
nPreviousBlockSize = nInDataSize;
|
||||
}
|
||||
|
||||
if (!bError && !feof(f_in) && nOriginalSize >= 1024 * 1024) {
|
||||
fprintf(stdout, "\r%lld => %lld (%g %%)", nOriginalSize, nCompressedSize, (double)(nCompressedSize * 100.0 / nOriginalSize));
|
||||
}
|
||||
}
|
||||
|
||||
unsigned char cFooter[3];
|
||||
|
||||
cFooter[0] = 0x00; /* Zero sized block */
|
||||
cFooter[1] = 0x00;
|
||||
cFooter[2] = 0x00;
|
||||
|
||||
if (!bError)
|
||||
bError = fwrite(cFooter, 1, 3, f_out) != 3;
|
||||
nCompressedSize += 3LL;
|
||||
|
||||
if (!bError && nVerbosity > 0) {
|
||||
nEndTime = lzsa_get_time();
|
||||
|
||||
double fDelta = ((double)(nEndTime - nStartTime)) / 1000000.0;
|
||||
double fSpeed = ((double)nOriginalSize / 1048576.0) / fDelta;
|
||||
fprintf(stdout, "\rCompressed '%s' in %g seconds, %.02g Mb/s, %lld into %lld bytes ==> %g %%\n",
|
||||
pszInFilename, fDelta, fSpeed, nOriginalSize, nCompressedSize, (double)(nCompressedSize * 100.0 / nOriginalSize));
|
||||
}
|
||||
|
||||
lzsa_compressor_destroy(&compressor);
|
||||
|
||||
free(pOutData);
|
||||
pOutData = NULL;
|
||||
|
||||
free(pInData);
|
||||
pInData = NULL;
|
||||
|
||||
fclose(f_out);
|
||||
f_out = NULL;
|
||||
|
||||
fclose(f_in);
|
||||
f_in = NULL;
|
||||
|
||||
if (bError) {
|
||||
fprintf(stderr, "\rcompression error for '%s'\n", pszInFilename);
|
||||
return 100;
|
||||
}
|
||||
else {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
static int lzsa_decompress(const char *pszInFilename, const char *pszOutFilename, int nVerbosity) {
|
||||
long long nStartTime = 0LL, nEndTime = 0LL;
|
||||
long long nOriginalSize = 0LL;
|
||||
|
||||
FILE *pInFile = fopen(pszInFilename, "rb");
|
||||
if (!pInFile) {
|
||||
fprintf(stderr, "error opening input file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
unsigned char cHeader[5];
|
||||
|
||||
memset(cHeader, 0, 5);
|
||||
|
||||
if (fread(cHeader, 1, 5, pInFile) != 5) {
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
fprintf(stderr, "error reading header in input file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
if (cHeader[0] != 0x7b ||
|
||||
cHeader[1] != 0x9e ||
|
||||
cHeader[2] != 0x0f ||
|
||||
cHeader[3] != 0xd7 ||
|
||||
cHeader[4] != 0) {
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
fprintf(stderr, "invalid magic number or format version in input file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
FILE *pOutFile = fopen(pszOutFilename, "wb");
|
||||
if (!pOutFile) {
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
fprintf(stderr, "error opening output file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
unsigned char *pInBlock;
|
||||
unsigned char *pOutData;
|
||||
|
||||
pInBlock = (unsigned char*)malloc(BLOCK_SIZE);
|
||||
if (!pInBlock) {
|
||||
fclose(pOutFile);
|
||||
pOutFile = NULL;
|
||||
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
fprintf(stderr, "error opening output file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
pOutData = (unsigned char*)malloc(BLOCK_SIZE * 2);
|
||||
if (!pOutData) {
|
||||
free(pInBlock);
|
||||
pInBlock = NULL;
|
||||
|
||||
fclose(pOutFile);
|
||||
pOutFile = NULL;
|
||||
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
fprintf(stderr, "error opening output file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
if (nVerbosity > 0) {
|
||||
nStartTime = lzsa_get_time();
|
||||
}
|
||||
|
||||
int nDecompressionError = 0;
|
||||
int nPrevDecompressedSize = 0;
|
||||
|
||||
while (!feof(pInFile) && !nDecompressionError) {
|
||||
unsigned char cBlockSize[3];
|
||||
|
||||
if (nPrevDecompressedSize != 0) {
|
||||
memcpy(pOutData + BLOCK_SIZE - nPrevDecompressedSize, pOutData + BLOCK_SIZE, nPrevDecompressedSize);
|
||||
}
|
||||
|
||||
if (fread(cBlockSize, 1, 3, pInFile) == 3) {
|
||||
unsigned int nBlockSize = ((unsigned int)cBlockSize[0]) |
|
||||
(((unsigned int)cBlockSize[1]) << 8) |
|
||||
(((unsigned int)cBlockSize[2]) << 16);
|
||||
bool bIsUncompressed = (nBlockSize & 0x800000) != 0;
|
||||
int nDecompressedSize = 0;
|
||||
|
||||
nBlockSize &= 0x7fffff;
|
||||
if (fread(pInBlock, 1, nBlockSize, pInFile) == nBlockSize) {
|
||||
if (bIsUncompressed) {
|
||||
memcpy(pOutData + BLOCK_SIZE, pInBlock, nBlockSize);
|
||||
nDecompressedSize = nBlockSize;
|
||||
}
|
||||
else {
|
||||
unsigned int nBlockOffs = 0;
|
||||
|
||||
nDecompressedSize = lzsa_expand_block(pInBlock, nBlockSize, pOutData, BLOCK_SIZE, BLOCK_SIZE);
|
||||
if (nDecompressedSize < 0) {
|
||||
nDecompressionError = nDecompressedSize;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (nDecompressedSize != 0) {
|
||||
nOriginalSize += (long long)nDecompressedSize;
|
||||
|
||||
fwrite(pOutData + BLOCK_SIZE, 1, nDecompressedSize, pOutFile);
|
||||
nPrevDecompressedSize = nDecompressedSize;
|
||||
nDecompressedSize = 0;
|
||||
}
|
||||
}
|
||||
else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
free(pOutData);
|
||||
pOutData = NULL;
|
||||
|
||||
free(pInBlock);
|
||||
pInBlock = NULL;
|
||||
|
||||
fclose(pOutFile);
|
||||
pOutFile = NULL;
|
||||
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
|
||||
if (nDecompressionError) {
|
||||
fprintf(stderr, "decompression error for '%s'\n", pszInFilename);
|
||||
return 100;
|
||||
}
|
||||
else {
|
||||
if (nVerbosity > 0) {
|
||||
nEndTime = lzsa_get_time();
|
||||
double fDelta = ((double)(nEndTime - nStartTime)) / 1000000.0;
|
||||
double fSpeed = ((double)nOriginalSize / 1048576.0) / fDelta;
|
||||
fprintf(stdout, "Decompressed '%s' in %g seconds, %g Mb/s\n",
|
||||
pszInFilename, fDelta, fSpeed);
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
static int lzsa_compare(const char *pszInFilename, const char *pszOutFilename, int nVerbosity) {
|
||||
long long nStartTime = 0LL, nEndTime = 0LL;
|
||||
long long nOriginalSize = 0LL;
|
||||
long long nKnownGoodSize = 0LL;
|
||||
|
||||
FILE *pInFile = fopen(pszInFilename, "rb");
|
||||
if (!pInFile) {
|
||||
fprintf(stderr, "error opening compressed input file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
unsigned char cHeader[5];
|
||||
|
||||
memset(cHeader, 0, 5);
|
||||
|
||||
if (fread(cHeader, 1, 5, pInFile) != 5) {
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
fprintf(stderr, "error reading header in compressed input file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
if (cHeader[0] != 0x7b ||
|
||||
cHeader[1] != 0x9e ||
|
||||
cHeader[2] != 0x0f ||
|
||||
cHeader[3] != 0xd7 ||
|
||||
cHeader[4] != 0) {
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
fprintf(stderr, "invalid magic number or format version in input file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
FILE *pOutFile = fopen(pszOutFilename, "rb");
|
||||
if (!pOutFile) {
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
fprintf(stderr, "error opening original uncompressed file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
unsigned char *pInBlock;
|
||||
unsigned char *pOutData;
|
||||
unsigned char *pCompareData;
|
||||
|
||||
pInBlock = (unsigned char*)malloc(BLOCK_SIZE);
|
||||
if (!pInBlock) {
|
||||
fclose(pOutFile);
|
||||
pOutFile = NULL;
|
||||
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
fprintf(stderr, "error opening output file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
pOutData = (unsigned char*)malloc(BLOCK_SIZE * 2);
|
||||
if (!pOutData) {
|
||||
free(pInBlock);
|
||||
pInBlock = NULL;
|
||||
|
||||
fclose(pOutFile);
|
||||
pOutFile = NULL;
|
||||
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
fprintf(stderr, "error opening output file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
pCompareData = (unsigned char*)malloc(BLOCK_SIZE);
|
||||
if (!pCompareData) {
|
||||
free(pOutData);
|
||||
pOutData = NULL;
|
||||
|
||||
free(pInBlock);
|
||||
pInBlock = NULL;
|
||||
|
||||
fclose(pOutFile);
|
||||
pOutFile = NULL;
|
||||
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
fprintf(stderr, "error opening output file\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
if (nVerbosity > 0) {
|
||||
nStartTime = lzsa_get_time();
|
||||
}
|
||||
|
||||
int nDecompressionError = 0;
|
||||
bool bComparisonError = false;
|
||||
int nPrevDecompressedSize = 0;
|
||||
|
||||
while (!feof(pInFile) && !feof(pOutFile) && !nDecompressionError && !bComparisonError) {
|
||||
unsigned char cBlockSize[3];
|
||||
|
||||
if (nPrevDecompressedSize != 0) {
|
||||
memcpy(pOutData + BLOCK_SIZE - nPrevDecompressedSize, pOutData + BLOCK_SIZE, nPrevDecompressedSize);
|
||||
}
|
||||
|
||||
int nBytesToCompare = (int)fread(pCompareData, 1, BLOCK_SIZE, pOutFile);
|
||||
|
||||
if (fread(cBlockSize, 1, 3, pInFile) == 3) {
|
||||
unsigned int nBlockSize = ((unsigned int)cBlockSize[0]) |
|
||||
(((unsigned int)cBlockSize[1]) << 8) |
|
||||
(((unsigned int)cBlockSize[2]) << 16);
|
||||
bool bIsUncompressed = (nBlockSize & 0x800000) != 0;
|
||||
int nDecompressedSize = 0;
|
||||
|
||||
nBlockSize &= 0x7fffff;
|
||||
if (fread(pInBlock, 1, nBlockSize, pInFile) == nBlockSize) {
|
||||
if (bIsUncompressed) {
|
||||
memcpy(pOutData + BLOCK_SIZE, pInBlock, nBlockSize);
|
||||
nDecompressedSize = nBlockSize;
|
||||
}
|
||||
else {
|
||||
unsigned int nBlockOffs = 0;
|
||||
|
||||
nDecompressedSize = lzsa_expand_block(pInBlock, nBlockSize, pOutData, BLOCK_SIZE, BLOCK_SIZE);
|
||||
if (nDecompressedSize < 0) {
|
||||
nDecompressionError = nDecompressedSize;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (nDecompressedSize == nBytesToCompare) {
|
||||
nKnownGoodSize = nOriginalSize;
|
||||
|
||||
nOriginalSize += (long long)nDecompressedSize;
|
||||
|
||||
if (memcmp(pOutData + BLOCK_SIZE, pCompareData, nBytesToCompare))
|
||||
bComparisonError = true;
|
||||
nPrevDecompressedSize = nDecompressedSize;
|
||||
nDecompressedSize = 0;
|
||||
}
|
||||
else {
|
||||
bComparisonError = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
free(pCompareData);
|
||||
pCompareData = NULL;
|
||||
|
||||
free(pOutData);
|
||||
pOutData = NULL;
|
||||
|
||||
free(pInBlock);
|
||||
pInBlock = NULL;
|
||||
|
||||
fclose(pOutFile);
|
||||
pOutFile = NULL;
|
||||
|
||||
fclose(pInFile);
|
||||
pInFile = NULL;
|
||||
|
||||
if (nDecompressionError) {
|
||||
fprintf(stderr, "decompression error for '%s'\n", pszInFilename);
|
||||
return 100;
|
||||
}
|
||||
else if (bComparisonError) {
|
||||
fprintf(stderr, "error comparing compressed file '%s' with original '%s' starting at %lld\n", pszInFilename, pszOutFilename, nKnownGoodSize);
|
||||
return 100;
|
||||
}
|
||||
else {
|
||||
if (nVerbosity > 0) {
|
||||
nEndTime = lzsa_get_time();
|
||||
double fDelta = ((double)(nEndTime - nStartTime)) / 1000000.0;
|
||||
double fSpeed = ((double)nOriginalSize / 1048576.0) / fDelta;
|
||||
fprintf(stdout, "Compared '%s' in %g seconds, %g Mb/s\n",
|
||||
pszInFilename, fDelta, fSpeed);
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
/*---------------------------------------------------------------------------*/
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
int i;
|
||||
const char *pszInFilename = NULL;
|
||||
const char *pszOutFilename = NULL;
|
||||
bool bArgsError = false;
|
||||
bool bCommandDefined = false;
|
||||
bool bVerifyCompression = false;
|
||||
char cCommand = 'z';
|
||||
int nVerbosity = 0;
|
||||
|
||||
for (i = 1; i < argc; i++) {
|
||||
if (!strcmp(argv[i], "-d")) {
|
||||
if (!bCommandDefined) {
|
||||
bCommandDefined = true;
|
||||
cCommand = 'd';
|
||||
}
|
||||
else
|
||||
bArgsError = true;
|
||||
}
|
||||
else if (!strcmp(argv[i], "-z")) {
|
||||
if (!bCommandDefined) {
|
||||
bCommandDefined = true;
|
||||
cCommand = 'z';
|
||||
}
|
||||
else
|
||||
bArgsError = true;
|
||||
}
|
||||
else if (!strcmp(argv[i], "-c")) {
|
||||
if (!bVerifyCompression) {
|
||||
bVerifyCompression = true;
|
||||
}
|
||||
else
|
||||
bArgsError = true;
|
||||
}
|
||||
else if (!strcmp(argv[i], "-v")) {
|
||||
if (nVerbosity == 0) {
|
||||
nVerbosity = 1;
|
||||
}
|
||||
else
|
||||
bArgsError = true;
|
||||
}
|
||||
else {
|
||||
if (!pszInFilename)
|
||||
pszInFilename = argv[i];
|
||||
else {
|
||||
if (!pszOutFilename)
|
||||
pszOutFilename = argv[i];
|
||||
else
|
||||
bArgsError = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (bArgsError || !pszInFilename || !pszOutFilename) {
|
||||
fprintf(stderr, "usage: %s [-c] [-d] [-v] <infile> <outfile>\n", argv[0]);
|
||||
fprintf(stderr, " -c: check resulting stream after compressing\n");
|
||||
fprintf(stderr, " -d: decompress (default: compress)\n");
|
||||
fprintf(stderr, " -v: be verbose\n");
|
||||
return 100;
|
||||
}
|
||||
|
||||
if (cCommand == 'z') {
|
||||
int nResult = lzsa_compress(pszInFilename, pszOutFilename, nVerbosity);
|
||||
if (nResult == 0 && bVerifyCompression) {
|
||||
nResult = lzsa_compare(pszOutFilename, pszInFilename, nVerbosity);
|
||||
}
|
||||
}
|
||||
else if (cCommand == 'd') {
|
||||
return lzsa_decompress(pszInFilename, pszOutFilename, nVerbosity);
|
||||
}
|
||||
else {
|
||||
return 100;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,543 @@
|
|||
/*
|
||||
* shrink.c - block compressor implementation
|
||||
*
|
||||
* The following copying information applies to this specific source code file:
|
||||
*
|
||||
* Written in 2019 by Emmanuel Marty <marty.emmanuel@gmail.com>
|
||||
* Portions written in 2014-2015 by Eric Biggers <ebiggers3@gmail.com>
|
||||
*
|
||||
* To the extent possible under law, the author(s) have dedicated all copyright
|
||||
* and related and neighboring rights to this software to the public domain
|
||||
* worldwide via the Creative Commons Zero 1.0 Universal Public Domain
|
||||
* Dedication (the "CC0").
|
||||
*
|
||||
* This software is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
* FOR A PARTICULAR PURPOSE. See the CC0 for more details.
|
||||
*
|
||||
* You should have received a copy of the CC0 along with this software; if not
|
||||
* see <http://creativecommons.org/publicdomain/zero/1.0/>.
|
||||
*/
|
||||
|
||||
/*
|
||||
* Uses the libdivsufsort library Copyright (c) 2003-2008 Yuta Mori
|
||||
*
|
||||
* Inspired by LZ4 by Yann Collet. https://github.com/lz4/lz4
|
||||
* With ideas from Lizard by Przemyslaw Skibinski and Yann Collet. https://github.com/inikep/lizard
|
||||
*/
|
||||
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include "divsufsort.h"
|
||||
#include "shrink.h"
|
||||
#include "format.h"
|
||||
|
||||
#define LCP_BITS 14
|
||||
#define LCP_MAX ((1<<LCP_BITS) - 1)
|
||||
#define LCP_SHIFT (31-LCP_BITS)
|
||||
#define LCP_MASK (LCP_MAX << LCP_SHIFT)
|
||||
#define POS_MASK ((1<<LCP_SHIFT) - 1)
|
||||
|
||||
#define NMATCHES_PER_OFFSET 4
|
||||
#define MATCHES_PER_OFFSET_SHIFT 2
|
||||
|
||||
#define LEAVE_ALONE_MATCH_SIZE 600
|
||||
|
||||
int lzsa_compressor_init(lsza_compressor *pCompressor, const int nMaxDataSize) {
|
||||
pCompressor->intervals = (int *)malloc(nMaxDataSize * sizeof(int));
|
||||
pCompressor->pos_data = NULL;
|
||||
pCompressor->open_intervals = NULL;
|
||||
pCompressor->match = NULL;
|
||||
|
||||
if (pCompressor->intervals) {
|
||||
pCompressor->pos_data = (int *)malloc(nMaxDataSize * sizeof(int));
|
||||
|
||||
if (pCompressor->pos_data) {
|
||||
pCompressor->open_intervals = (int *)malloc((LCP_MAX + 1) * sizeof(int));
|
||||
|
||||
if (pCompressor->open_intervals) {
|
||||
pCompressor->match = (lzsa_match *)malloc(nMaxDataSize * NMATCHES_PER_OFFSET * sizeof(lzsa_match));
|
||||
|
||||
if (pCompressor->match)
|
||||
return 0;
|
||||
|
||||
free(pCompressor->open_intervals);
|
||||
pCompressor->open_intervals = NULL;
|
||||
}
|
||||
|
||||
free(pCompressor->pos_data);
|
||||
pCompressor->pos_data = NULL;
|
||||
}
|
||||
|
||||
free(pCompressor->intervals);
|
||||
pCompressor->intervals = NULL;
|
||||
}
|
||||
|
||||
return 100;
|
||||
}
|
||||
|
||||
void lzsa_compressor_destroy(lsza_compressor *pCompressor) {
|
||||
if (pCompressor->match) {
|
||||
free(pCompressor->match);
|
||||
pCompressor->match = NULL;
|
||||
}
|
||||
|
||||
if (pCompressor->open_intervals) {
|
||||
free(pCompressor->open_intervals);
|
||||
pCompressor->open_intervals = NULL;
|
||||
}
|
||||
|
||||
if (pCompressor->pos_data) {
|
||||
free(pCompressor->pos_data);
|
||||
pCompressor->pos_data = NULL;
|
||||
}
|
||||
|
||||
if (pCompressor->intervals) {
|
||||
free(pCompressor->intervals);
|
||||
pCompressor->intervals = NULL;
|
||||
}
|
||||
}
|
||||
|
||||
static int lzsa_build_suffix_array(lsza_compressor *pCompressor, const unsigned char *pInWindow, const int nInWindowSize) {
|
||||
int *intervals = pCompressor->intervals;
|
||||
|
||||
/* Build suffix array from input data */
|
||||
if (divsufsort(pInWindow, (saidx_t*)intervals, nInWindowSize) != 0) {
|
||||
return 100;
|
||||
}
|
||||
|
||||
int *PLCP = pCompressor->pos_data;
|
||||
int *Phi = PLCP;
|
||||
int nCurLen = 0;
|
||||
int i;
|
||||
|
||||
/* Compute the permuted LCP first (Kärkkäinen method) */
|
||||
Phi[intervals[0]] = -1;
|
||||
for (i = 1; i < nInWindowSize; i++)
|
||||
Phi[intervals[i]] = intervals[i - 1];
|
||||
for (i = 0; i < nInWindowSize; i++) {
|
||||
if (Phi[i] == -1) {
|
||||
PLCP[i] = 0;
|
||||
continue;
|
||||
}
|
||||
int nMaxLen = (i > Phi[i]) ? (nInWindowSize - i) : (nInWindowSize - Phi[i]);
|
||||
while (nCurLen < nMaxLen && pInWindow[i + nCurLen] == pInWindow[Phi[i] + nCurLen]) nCurLen++;
|
||||
PLCP[i] = nCurLen;
|
||||
if (nCurLen > 0)
|
||||
nCurLen--;
|
||||
}
|
||||
|
||||
/* Rotate permuted LCP into the LCP. This has better cache locality than the direct Kasai LCP method */
|
||||
intervals[0] &= POS_MASK;
|
||||
for (i = 1; i < nInWindowSize; i++) {
|
||||
int nIndex = intervals[i] & POS_MASK;
|
||||
int nLen = PLCP[nIndex];
|
||||
if (nLen < MIN_MATCH_SIZE)
|
||||
nLen = 0;
|
||||
if (nLen > LCP_MAX)
|
||||
nLen = LCP_MAX;
|
||||
intervals[i] = nIndex | (nLen << LCP_SHIFT);
|
||||
}
|
||||
|
||||
/**
|
||||
* Build intervals for finding matches
|
||||
*
|
||||
* Methodology and code fragment taken from wimlib (CC0 license):
|
||||
* https://wimlib.net/git/?p=wimlib;a=blob_plain;f=src/lcpit_matchfinder.c;h=a2d6a1e0cd95200d1f3a5464d8359d5736b14cbe;hb=HEAD
|
||||
*/
|
||||
int * const SA_and_LCP = intervals;
|
||||
int *pos_data = pCompressor->pos_data;
|
||||
int next_interval_idx;
|
||||
int *top = pCompressor->open_intervals;
|
||||
int prev_pos = SA_and_LCP[0] & POS_MASK;
|
||||
|
||||
*top = 0;
|
||||
intervals[0] = 0;
|
||||
next_interval_idx = 1;
|
||||
|
||||
for (int r = 1; r < nInWindowSize; r++) {
|
||||
const int next_pos = SA_and_LCP[r] & POS_MASK;
|
||||
const int next_lcp = SA_and_LCP[r] & LCP_MASK;
|
||||
const int top_lcp = *top & LCP_MASK;
|
||||
|
||||
if (next_lcp == top_lcp) {
|
||||
/* Continuing the deepest open interval */
|
||||
pos_data[prev_pos] = *top;
|
||||
}
|
||||
else if (next_lcp > top_lcp) {
|
||||
/* Opening a new interval */
|
||||
*++top = next_lcp | next_interval_idx++;
|
||||
pos_data[prev_pos] = *top;
|
||||
}
|
||||
else {
|
||||
/* Closing the deepest open interval */
|
||||
pos_data[prev_pos] = *top;
|
||||
for (;;) {
|
||||
const int closed_interval_idx = *top-- & POS_MASK;
|
||||
const int superinterval_lcp = *top & LCP_MASK;
|
||||
|
||||
if (next_lcp == superinterval_lcp) {
|
||||
/* Continuing the superinterval */
|
||||
intervals[closed_interval_idx] = *top;
|
||||
break;
|
||||
}
|
||||
else if (next_lcp > superinterval_lcp) {
|
||||
/* Creating a new interval that is a
|
||||
* superinterval of the one being
|
||||
* closed, but still a subinterval of
|
||||
* its superinterval */
|
||||
*++top = next_lcp | next_interval_idx++;
|
||||
intervals[closed_interval_idx] = *top;
|
||||
break;
|
||||
}
|
||||
else {
|
||||
/* Also closing the superinterval */
|
||||
intervals[closed_interval_idx] = *top;
|
||||
}
|
||||
}
|
||||
}
|
||||
prev_pos = next_pos;
|
||||
}
|
||||
|
||||
/* Close any still-open intervals. */
|
||||
pos_data[prev_pos] = *top;
|
||||
for (; top > pCompressor->open_intervals; top--)
|
||||
intervals[*top & POS_MASK] = *(top - 1);
|
||||
|
||||
/* Success */
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int lzsa_find_matches_at(lsza_compressor *pCompressor, const int nOffset, lzsa_match *pMatches, const int nMaxMatches) {
|
||||
int *intervals = pCompressor->intervals;
|
||||
int *pos_data = pCompressor->pos_data;
|
||||
int ref;
|
||||
int super_ref;
|
||||
int match_pos;
|
||||
lzsa_match *matchptr;
|
||||
|
||||
/**
|
||||
* Find matches using intervals
|
||||
*
|
||||
* Taken from wimlib (CC0 license):
|
||||
* https://wimlib.net/git/?p=wimlib;a=blob_plain;f=src/lcpit_matchfinder.c;h=a2d6a1e0cd95200d1f3a5464d8359d5736b14cbe;hb=HEAD
|
||||
*/
|
||||
|
||||
/* Get the deepest lcp-interval containing the current suffix. */
|
||||
ref = pos_data[nOffset];
|
||||
|
||||
pos_data[nOffset] = 0;
|
||||
|
||||
/* Ascend until we reach a visited interval, the root, or a child of the
|
||||
* root. Link unvisited intervals to the current suffix as we go. */
|
||||
while ((super_ref = intervals[ref & POS_MASK]) & LCP_MASK) {
|
||||
intervals[ref & POS_MASK] = nOffset;
|
||||
ref = super_ref;
|
||||
}
|
||||
|
||||
if (super_ref == 0) {
|
||||
/* In this case, the current interval may be any of:
|
||||
* (1) the root;
|
||||
* (2) an unvisited child of the root;
|
||||
* (3) an interval last visited by suffix 0
|
||||
*
|
||||
* We could avoid the ambiguity with (3) by using an lcp
|
||||
* placeholder value other than 0 to represent "visited", but
|
||||
* it's fastest to use 0. So we just don't allow matches with
|
||||
* position 0. */
|
||||
|
||||
if (ref != 0) /* Not the root? */
|
||||
intervals[ref & POS_MASK] = nOffset;
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Ascend indirectly via pos_data[] links. */
|
||||
match_pos = super_ref;
|
||||
matchptr = pMatches;
|
||||
for (;;) {
|
||||
while ((super_ref = pos_data[match_pos]) > ref)
|
||||
match_pos = intervals[super_ref & POS_MASK];
|
||||
intervals[ref & POS_MASK] = nOffset;
|
||||
pos_data[match_pos] = ref;
|
||||
|
||||
if ((matchptr - pMatches) < nMaxMatches) {
|
||||
int nMatchOffset = nOffset - match_pos;
|
||||
|
||||
if (nMatchOffset <= MAX_OFFSET) {
|
||||
matchptr->length = (unsigned short)(ref >> LCP_SHIFT);
|
||||
matchptr->offset = (unsigned short)nMatchOffset;
|
||||
matchptr++;
|
||||
}
|
||||
}
|
||||
|
||||
if (super_ref == 0)
|
||||
break;
|
||||
ref = super_ref;
|
||||
match_pos = intervals[ref & POS_MASK];
|
||||
}
|
||||
|
||||
return (int)(matchptr - pMatches);
|
||||
}
|
||||
|
||||
static void lzsa_skip_matches(lsza_compressor *pCompressor, const int nStartOffset, const int nEndOffset) {
|
||||
lzsa_match match;
|
||||
int i;
|
||||
|
||||
for (i = nStartOffset; i < nEndOffset; i++) {
|
||||
lzsa_find_matches_at(pCompressor, i, &match, 0);
|
||||
}
|
||||
}
|
||||
|
||||
static void lzsa_find_all_matches(lsza_compressor *pCompressor, const int nStartOffset, const int nEndOffset) {
|
||||
lzsa_match *pMatch = pCompressor->match + (nStartOffset << MATCHES_PER_OFFSET_SHIFT);
|
||||
int i;
|
||||
|
||||
for (i = nStartOffset; i < nEndOffset; i++) {
|
||||
int nMatches = lzsa_find_matches_at(pCompressor, i, pMatch, NMATCHES_PER_OFFSET);
|
||||
int m;
|
||||
|
||||
for (m = 0; m < NMATCHES_PER_OFFSET; m++) {
|
||||
if (nMatches <= m || i >= (nEndOffset - 11)) {
|
||||
pMatch->length = 0;
|
||||
pMatch->offset = 0;
|
||||
}
|
||||
else {
|
||||
int nMaxLen = (nEndOffset - 5) - i;
|
||||
if (nMaxLen < 0)
|
||||
nMaxLen = 0;
|
||||
if (pMatch->length > nMaxLen)
|
||||
pMatch->length = (unsigned short)nMaxLen;
|
||||
}
|
||||
|
||||
pMatch++;
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
static inline int lzsa_get_literals_varlen_size(const int nLength) {
|
||||
if (nLength < LITERALS_RUN_LEN) {
|
||||
return 0;
|
||||
}
|
||||
else {
|
||||
if (nLength < (LITERALS_RUN_LEN + 254))
|
||||
return 1;
|
||||
else {
|
||||
if (nLength < (LITERALS_RUN_LEN + 510))
|
||||
return 2;
|
||||
else
|
||||
return 3;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static inline int lzsa_write_literals_varlen(unsigned char *pOutData, int nOutOffset, int nLength) {
|
||||
if (nLength >= LITERALS_RUN_LEN) {
|
||||
nLength -= LITERALS_RUN_LEN;
|
||||
|
||||
if (nLength < 254)
|
||||
pOutData[nOutOffset++] = nLength;
|
||||
else {
|
||||
if (nLength < 510) {
|
||||
pOutData[nOutOffset++] = 254;
|
||||
pOutData[nOutOffset++] = nLength - 254;
|
||||
}
|
||||
else {
|
||||
pOutData[nOutOffset++] = 255;
|
||||
pOutData[nOutOffset++] = (nLength - 255) & 0xff;
|
||||
pOutData[nOutOffset++] = ((nLength - 255) >> 8) & 0xff;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nOutOffset;
|
||||
}
|
||||
|
||||
static inline int lzsa_get_match_varlen_size(const int nLength) {
|
||||
if (nLength < MATCH_RUN_LEN) {
|
||||
return 0;
|
||||
}
|
||||
else {
|
||||
if (nLength < (MATCH_RUN_LEN + 254))
|
||||
return 1;
|
||||
else {
|
||||
if (nLength < (MATCH_RUN_LEN + 510))
|
||||
return 2;
|
||||
else
|
||||
return 3;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static inline int lzsa_write_match_varlen(unsigned char *pOutData, int nOutOffset, int nLength) {
|
||||
if (nLength >= MATCH_RUN_LEN) {
|
||||
nLength -= MATCH_RUN_LEN;
|
||||
|
||||
if (nLength < 254)
|
||||
pOutData[nOutOffset++] = nLength;
|
||||
else {
|
||||
if (nLength < 510) {
|
||||
pOutData[nOutOffset++] = 254;
|
||||
pOutData[nOutOffset++] = nLength - 254;
|
||||
}
|
||||
else {
|
||||
pOutData[nOutOffset++] = 255;
|
||||
pOutData[nOutOffset++] = (nLength - 255) & 0xff;
|
||||
pOutData[nOutOffset++] = ((nLength - 255) >> 8) & 0xff;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nOutOffset;
|
||||
}
|
||||
|
||||
static void lzsa_optimize_matches(lsza_compressor *pCompressor, const int nStartOffset, const int nEndOffset) {
|
||||
int *cost = pCompressor->pos_data; /* Reuse */
|
||||
int nLastLiteralsOffset;
|
||||
int i;
|
||||
|
||||
cost[nEndOffset - 1] = 1;
|
||||
nLastLiteralsOffset = nEndOffset - 1;
|
||||
|
||||
for (i = nEndOffset - 2; i != (nStartOffset - 1); i--) {
|
||||
int nBestCost, nBestMatchLen, nBestMatchOffset;
|
||||
|
||||
int nLiteralsLen = nLastLiteralsOffset - i;
|
||||
nBestCost = 1 + cost[i + 1] + lzsa_get_literals_varlen_size(nLiteralsLen);
|
||||
nBestMatchLen = 0;
|
||||
nBestMatchOffset = 0;
|
||||
|
||||
lzsa_match *pMatch = pCompressor->match + (i << MATCHES_PER_OFFSET_SHIFT);
|
||||
int m;
|
||||
|
||||
for (m = 0; m < NMATCHES_PER_OFFSET; m++) {
|
||||
int nMatchOffsetSize = (pMatch[m].offset <= 256) ? 1 : 2;
|
||||
|
||||
if (pMatch[m].length >= LEAVE_ALONE_MATCH_SIZE) {
|
||||
int nCurCost;
|
||||
int nMatchLen = pMatch[m].length;
|
||||
int nRemainingLiteralsLen = nLastLiteralsOffset - (i + nMatchLen);
|
||||
|
||||
if (nRemainingLiteralsLen < 0) nRemainingLiteralsLen = 0;
|
||||
|
||||
nCurCost = 1 + lzsa_get_literals_varlen_size(nRemainingLiteralsLen) + nMatchOffsetSize + lzsa_get_match_varlen_size(nMatchLen - MIN_MATCH_SIZE);
|
||||
if ((i + nMatchLen) < nEndOffset)
|
||||
nCurCost += cost[i + nMatchLen];
|
||||
|
||||
if (nBestCost >= nCurCost) {
|
||||
nBestCost = nCurCost;
|
||||
nBestMatchLen = nMatchLen;
|
||||
nBestMatchOffset = pMatch[m].offset;
|
||||
}
|
||||
}
|
||||
else {
|
||||
if (pMatch[m].length >= MIN_MATCH_SIZE) {
|
||||
int k;
|
||||
|
||||
for (k = MIN_MATCH_SIZE; k <= pMatch[m].length; k++) {
|
||||
int nCurCost;
|
||||
int nRemainingLiteralsLen = nLastLiteralsOffset - (i + k);
|
||||
|
||||
if (nRemainingLiteralsLen < 0) nRemainingLiteralsLen = 0;
|
||||
|
||||
nCurCost = 1 + lzsa_get_literals_varlen_size(nRemainingLiteralsLen) + nMatchOffsetSize + lzsa_get_match_varlen_size(k - MIN_MATCH_SIZE);
|
||||
if ((i + k) < nEndOffset)
|
||||
nCurCost += cost[i + k];
|
||||
|
||||
if (nBestCost >= nCurCost) {
|
||||
nBestCost = nCurCost;
|
||||
nBestMatchLen = k;
|
||||
nBestMatchOffset = pMatch[m].offset;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (nBestMatchLen >= MIN_MATCH_SIZE)
|
||||
nLastLiteralsOffset = i;
|
||||
|
||||
cost[i] = nBestCost;
|
||||
pMatch->length = nBestMatchLen;
|
||||
pMatch->offset = nBestMatchOffset;
|
||||
}
|
||||
}
|
||||
|
||||
static int lzsa_write_block(lsza_compressor *pCompressor, const unsigned char *pInWindow, const int nStartOffset, const int nEndOffset, unsigned char *pOutData, const int nMaxOutDataSize) {
|
||||
int i;
|
||||
int nNumLiterals = 0;
|
||||
int nInFirstLiteralOffset = 0;
|
||||
int nOutOffset = 0;
|
||||
|
||||
for (i = nStartOffset; i < nEndOffset; ) {
|
||||
lzsa_match *pMatch = pCompressor->match + (i << MATCHES_PER_OFFSET_SHIFT);
|
||||
|
||||
if (pMatch->length >= MIN_MATCH_SIZE) {
|
||||
int nMatchOffset = pMatch->offset;
|
||||
int nMatchLen = pMatch->length;
|
||||
int nEncodedMatchLen = nMatchLen - MIN_MATCH_SIZE;
|
||||
int nNibbleLiteralsLen = (nNumLiterals >= LITERALS_RUN_LEN) ? LITERALS_RUN_LEN : nNumLiterals;
|
||||
int nNibbleMatchLen = (nEncodedMatchLen >= MATCH_RUN_LEN) ? MATCH_RUN_LEN : nEncodedMatchLen;
|
||||
int nNibbleLongOffset = (nMatchOffset <= 256) ? 0x00 : 0x80;
|
||||
int nTokenSize = 1 /* nibble */ + lzsa_get_literals_varlen_size(nNumLiterals) + nNumLiterals + (nNibbleLongOffset ? 2 : 1) /* match offset */ + lzsa_get_match_varlen_size(nEncodedMatchLen);
|
||||
|
||||
if ((nOutOffset + nTokenSize) > nMaxOutDataSize)
|
||||
return -1;
|
||||
if (nMatchOffset < MIN_OFFSET || nMatchOffset > MAX_OFFSET)
|
||||
return -1;
|
||||
|
||||
pOutData[nOutOffset++] = nNibbleLongOffset | (nNibbleLiteralsLen << 4) | nNibbleMatchLen;
|
||||
nOutOffset = lzsa_write_literals_varlen(pOutData, nOutOffset, nNumLiterals);
|
||||
|
||||
if (nNumLiterals != 0) {
|
||||
memcpy(pOutData + nOutOffset, pInWindow + nInFirstLiteralOffset, nNumLiterals);
|
||||
nOutOffset += nNumLiterals;
|
||||
nNumLiterals = 0;
|
||||
}
|
||||
|
||||
pOutData[nOutOffset++] = (nMatchOffset - 1) & 0xff;
|
||||
if (nNibbleLongOffset)
|
||||
pOutData[nOutOffset++] = (nMatchOffset - 1) >> 8;
|
||||
|
||||
nOutOffset = lzsa_write_match_varlen(pOutData, nOutOffset, nEncodedMatchLen);
|
||||
i += nMatchLen;
|
||||
}
|
||||
else {
|
||||
if (nNumLiterals == 0)
|
||||
nInFirstLiteralOffset = i;
|
||||
nNumLiterals++;
|
||||
i++;
|
||||
}
|
||||
}
|
||||
|
||||
if (nNumLiterals != 0) {
|
||||
int nNibbleLiteralsLen = (nNumLiterals >= LITERALS_RUN_LEN) ? LITERALS_RUN_LEN : nNumLiterals;
|
||||
int nTokenSize = 1 /* nibble */ + lzsa_get_literals_varlen_size(nNumLiterals) + nNumLiterals;
|
||||
|
||||
if ((nOutOffset + nTokenSize) > nMaxOutDataSize)
|
||||
return -1;
|
||||
|
||||
pOutData[nOutOffset++] = (nNibbleLiteralsLen << 4);
|
||||
nOutOffset = lzsa_write_literals_varlen(pOutData, nOutOffset, nNumLiterals);
|
||||
|
||||
if (nNumLiterals != 0) {
|
||||
memcpy(pOutData + nOutOffset, pInWindow + nInFirstLiteralOffset, nNumLiterals);
|
||||
nOutOffset += nNumLiterals;
|
||||
nNumLiterals = 0;
|
||||
}
|
||||
}
|
||||
|
||||
return nOutOffset;
|
||||
}
|
||||
|
||||
int lzsa_shrink_block(lsza_compressor *pCompressor, const unsigned char *pInWindow, const int nPreviousBlockSize, const int nInDataSize, unsigned char *pOutData, const int nMaxOutDataSize) {
|
||||
lzsa_build_suffix_array(pCompressor, pInWindow, nPreviousBlockSize + nInDataSize);
|
||||
if (nPreviousBlockSize) {
|
||||
lzsa_skip_matches(pCompressor, 0, nPreviousBlockSize);
|
||||
}
|
||||
lzsa_find_all_matches(pCompressor, nPreviousBlockSize, nPreviousBlockSize + nInDataSize);
|
||||
lzsa_optimize_matches(pCompressor, nPreviousBlockSize, nPreviousBlockSize + nInDataSize);
|
||||
|
||||
return lzsa_write_block(pCompressor, pInWindow, nPreviousBlockSize, nPreviousBlockSize + nInDataSize, pOutData, nMaxOutDataSize);
|
||||
}
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
/*
|
||||
* shrink.h - block compressor definitions
|
||||
*
|
||||
* Copyright (C) 2019 Emmanuel Marty
|
||||
*
|
||||
* This software is provided 'as-is', without any express or implied
|
||||
* warranty. In no event will the authors be held liable for any damages
|
||||
* arising from the use of this software.
|
||||
*
|
||||
* Permission is granted to anyone to use this software for any purpose,
|
||||
* including commercial applications, and to alter it and redistribute it
|
||||
* freely, subject to the following restrictions:
|
||||
*
|
||||
* 1. The origin of this software must not be misrepresented; you must not
|
||||
* claim that you wrote the original software. If you use this software
|
||||
* in a product, an acknowledgment in the product documentation would be
|
||||
* appreciated but is not required.
|
||||
* 2. Altered source versions must be plainly marked as such, and must not be
|
||||
* misrepresented as being the original software.
|
||||
* 3. This notice may not be removed or altered from any source distribution.
|
||||
*/
|
||||
|
||||
#ifndef _SHRINK_H
|
||||
#define _SHRINK_H
|
||||
|
||||
typedef struct {
|
||||
unsigned short length;
|
||||
unsigned short offset;
|
||||
} lzsa_match;
|
||||
|
||||
typedef struct {
|
||||
int *intervals;
|
||||
int *pos_data;
|
||||
int *open_intervals;
|
||||
lzsa_match *match;
|
||||
} lsza_compressor;
|
||||
|
||||
int lzsa_compressor_init(lsza_compressor *pCompressor, const int nMaxDataSize);
|
||||
void lzsa_compressor_destroy(lsza_compressor *pCompressor);
|
||||
int lzsa_shrink_block(lsza_compressor *pCompressor, const unsigned char *pInWindow, const int nPreviousBlockSize, const int nInDataSize, unsigned char *pOutData, const int nMaxOutDataSize);
|
||||
|
||||
#endif /* _SHRINK_H */
|
||||
Loading…
Reference in New Issue