Apple-2-Tools/apple2ix
1
0
Fork 0
mirror of https://github.com/mauiaaron/apple2.git synced 2025-01-10 06:31:13 +00:00
Code Issues Projects Releases Wiki Activity

Include jsmn JSON parsing code; add our own wrapper for processing JSON daata from files

Browse Source
This commit is contained in:
Aaron Culliney 2015-08-28 22:09:35 -07:00
parent 730a09880e
commit 0bd9591348
7 changed files with 1419 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
externals/jsmn/LICENSE vendored Normal file
View File

@ -0,0 +1,20 @@
Copyright (c) 2010 Serge A. Zaitsev
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

167
externals/jsmn/README.md vendored Normal file
View File

@ -0,0 +1,167 @@
JSMN
====
jsmn (pronounced like 'jasmine') is a minimalistic JSON parser in C. It can be
easily integrated into resource-limited or embedded projects.
You can find more information about JSON format at [json.org][1]
Library sources are available at [bitbucket.org/zserge/jsmn][2]
The web page with some information about jsmn can be found at
[http://zserge.com/jsmn.html][3]
Philosophy
----------
Most JSON parsers offer you a bunch of functions to load JSON data, parse it
and extract any value by its name. jsmn proves that checking the correctness of
every JSON packet or allocating temporary objects to store parsed JSON fields
often is an overkill.
JSON format itself is extremely simple, so why should we complicate it?
jsmn is designed to be **robust** (it should work fine even with erroneous
data), **fast** (it should parse data on the fly), **portable** (no superfluous
dependencies or non-standard C extensions). An of course, **simplicity** is a
key feature - simple code style, simple algorithm, simple integration into
other projects.
Features
--------
* compatible with C89
* no dependencies (even libc!)
* highly portable (tested on x86/amd64, ARM, AVR)
* about 200 lines of code
* extremely small code footprint
* API contains only 2 functions
* no dynamic memory allocation
* incremental single-pass parsing
* library code is covered with unit-tests
Design
------
The rudimentary jsmn object is a **token**. Let's consider a JSON string:
'{ "name" : "Jack", "age" : 27 }'
It holds the following tokens:
* Object: `{ "name" : "Jack", "age" : 27}` (the whole object)
* Strings: `"name"`, `"Jack"`, `"age"` (keys and some values)
* Number: `27`
In jsmn, tokens do not hold any data, but point to token boundaries in JSON
string instead. In the example above jsmn will create tokens like: Object
[0..31], String [3..7], String [12..16], String [20..23], Number [27..29].
Every jsmn token has a type, which indicates the type of corresponding JSON
token. jsmn supports the following token types:
* Object - a container of key-value pairs, e.g.:
`{ "foo":"bar", "x":0.3 }`
* Array - a sequence of values, e.g.:
`[ 1, 2, 3 ]`
* String - a quoted sequence of chars, e.g.: `"foo"`
* Primitive - a number, a boolean (`true`, `false`) or `null`
Besides start/end positions, jsmn tokens for complex types (like arrays
or objects) also contain a number of child items, so you can easily follow
object hierarchy.
This approach provides enough information for parsing any JSON data and makes
it possible to use zero-copy techniques.
Install
-------
To clone the repository you should have mercurial installed. Just run:
$ hg clone http://bitbucket.org/zserge/jsmn jsmn
Repository layout is simple: jsmn.c and jsmn.h are library files, tests are in
the jsmn\_test.c, you will also find README, LICENSE and Makefile files inside.
To build the library, run `make`. It is also recommended to run `make test`.
Let me know, if some tests fail.
If build was successful, you should get a `libjsmn.a` library.
The header file you should include is called `"jsmn.h"`.
API
---
Token types are described by `jsmntype_t`:
typedef enum {
JSMN_PRIMITIVE = 0,
JSMN_OBJECT = 1,
JSMN_ARRAY = 2,
JSMN_STRING = 3
} jsmntype_t;
**Note:** Unlike JSON data types, primitive tokens are not divided into
numbers, booleans and null, because one can easily tell the type using the
first character:
* <code>'t', 'f'</code> - boolean
* <code>'n'</code> - null
* <code>'-', '0'..'9'</code> - number
Token is an object of `jsmntok_t` type:
typedef struct {
jsmntype_t type; // Token type
int start; // Token start position
int end; // Token end position
int size; // Number of child (nested) tokens
} jsmntok_t;
**Note:** string tokens point to the first character after
the opening quote and the previous symbol before final quote. This was made
to simplify string extraction from JSON data.
All job is done by `jsmn_parser` object. You can initialize a new parser using:
jsmn_parser parser;
jsmntok_t tokens[10];
jsmn_init(&parser);
// js - pointer to JSON string
// tokens - an array of tokens available
// 10 - number of tokens available
jsmn_parse(&parser, js, tokens, 10);
This will create a parser, and then it tries to parse up to 10 JSON tokens from
the `js` string.
A non-negative reutrn value of `jsmn_parse` is the number of tokens actually
used by the parser.
Passing NULL instead of the tokens array would not store parsing results, but
instead the function will return the value of tokens needed to parse the given
string. This can be useful if you don't know yet how many tokens to allocate.
If something goes wrong, you will get an error. Error will be one of these:
* `JSMN_ERROR_INVAL` - bad token, JSON string is corrupted
* `JSMN_ERROR_NOMEM` - not enough tokens, JSON string is too large
* `JSMN_ERROR_PART` - JSON string is too short, expecting more JSON data
If you get `JSON_ERROR_NOMEM`, you can re-allocate more tokens and call
`jsmn_parse` once more. If you read json data from the stream, you can
periodically call `jsmn_parse` and check if return value is `JSON_ERROR_PART`.
You will get this error until you reach the end of JSON data.
Other info
----------
This software is distributed under [MIT license](http://www.opensource.org/licenses/mit-license.php),
so feel free to integrate it in your commercial products.
[1]: http://www.json.org/
[2]: https://bitbucket.org/zserge/jsmn/wiki/Home
[3]: http://zserge.com/jsmn.html

311
externals/jsmn/jsmn.c vendored Normal file
View File

@ -0,0 +1,311 @@
#include <stdlib.h>
#include "jsmn.h"
/**
* Allocates a fresh unused token from the token pull.
*/
static jsmntok_t *jsmn_alloc_token(jsmn_parser *parser,
jsmntok_t *tokens, size_t num_tokens) {
jsmntok_t *tok;
if (parser->toknext >= num_tokens) {
return NULL;
}
tok = &tokens[parser->toknext++];
tok->start = tok->end = -1;
tok->size = 0;
#ifdef JSMN_PARENT_LINKS
tok->parent = -1;
#endif
return tok;
}
/**
* Fills token type and boundaries.
*/
static void jsmn_fill_token(jsmntok_t *token, jsmntype_t type,
int start, int end) {
token->type = type;
token->start = start;
token->end = end;
token->size = 0;
}
/**
* Fills next available token with JSON primitive.
*/
static jsmnerr_t jsmn_parse_primitive(jsmn_parser *parser, const char *js,
size_t len, jsmntok_t *tokens, size_t num_tokens) {
jsmntok_t *token;
int start;
start = parser->pos;
for (; parser->pos < len && js[parser->pos] != '\0'; parser->pos++) {
switch (js[parser->pos]) {
#ifndef JSMN_STRICT
/* In strict mode primitive must be followed by "," or "}" or "]" */
case ':':
#endif
case '\t' : case '\r' : case '\n' : case ' ' :
case ',' : case ']' : case '}' :
goto found;
}
if (js[parser->pos] < 32 || js[parser->pos] >= 127) {
parser->pos = start;
return JSMN_ERROR_INVAL;
}
}
#ifdef JSMN_STRICT
/* In strict mode primitive must be followed by a comma/object/array */
parser->pos = start;
return JSMN_ERROR_PART;
#endif
found:
if (tokens == NULL) {
parser->pos--;
return 0;
}
token = jsmn_alloc_token(parser, tokens, num_tokens);
if (token == NULL) {
parser->pos = start;
return JSMN_ERROR_NOMEM;
}
jsmn_fill_token(token, JSMN_PRIMITIVE, start, parser->pos);
#ifdef JSMN_PARENT_LINKS
token->parent = parser->toksuper;
#endif
parser->pos--;
return 0;
}
/**
* Filsl next token with JSON string.
*/
static jsmnerr_t jsmn_parse_string(jsmn_parser *parser, const char *js,
size_t len, jsmntok_t *tokens, size_t num_tokens) {
jsmntok_t *token;
int start = parser->pos;
parser->pos++;
/* Skip starting quote */
for (; parser->pos < len && js[parser->pos] != '\0'; parser->pos++) {
char c = js[parser->pos];
/* Quote: end of string */
if (c == '\"') {
if (tokens == NULL) {
return 0;
}
token = jsmn_alloc_token(parser, tokens, num_tokens);
if (token == NULL) {
parser->pos = start;
return JSMN_ERROR_NOMEM;
}
jsmn_fill_token(token, JSMN_STRING, start+1, parser->pos);
#ifdef JSMN_PARENT_LINKS
token->parent = parser->toksuper;
#endif
return 0;
}
/* Backslash: Quoted symbol expected */
if (c == '\\' && parser->pos + 1 < len) {
int i;
parser->pos++;
switch (js[parser->pos]) {
/* Allowed escaped symbols */
case '\"': case '/' : case '\\' : case 'b' :
case 'f' : case 'r' : case 'n' : case 't' :
break;
/* Allows escaped symbol \uXXXX */
case 'u':
parser->pos++;
for(i = 0; i < 4 && parser->pos < len && js[parser->pos] != '\0'; i++) {
/* If it isn't a hex character we have an error */
if(!((js[parser->pos] >= 48 && js[parser->pos] <= 57) || /* 0-9 */
(js[parser->pos] >= 65 && js[parser->pos] <= 70) || /* A-F */
(js[parser->pos] >= 97 && js[parser->pos] <= 102))) { /* a-f */
parser->pos = start;
return JSMN_ERROR_INVAL;
}
parser->pos++;
}
parser->pos--;
break;
/* Unexpected symbol */
default:
parser->pos = start;
return JSMN_ERROR_INVAL;
}
}
}
parser->pos = start;
return JSMN_ERROR_PART;
}
/**
* Parse JSON string and fill tokens.
*/
jsmnerr_t jsmn_parse(jsmn_parser *parser, const char *js, size_t len,
jsmntok_t *tokens, unsigned int num_tokens) {
jsmnerr_t r;
int i;
jsmntok_t *token;
int count = 0;
for (; parser->pos < len && js[parser->pos] != '\0'; parser->pos++) {
char c;
jsmntype_t type;
c = js[parser->pos];
switch (c) {
case '{': case '[':
count++;
if (tokens == NULL) {
break;
}
token = jsmn_alloc_token(parser, tokens, num_tokens);
if (token == NULL)
return JSMN_ERROR_NOMEM;
if (parser->toksuper != -1) {
tokens[parser->toksuper].size++;
#ifdef JSMN_PARENT_LINKS
token->parent = parser->toksuper;
#endif
}
token->type = (c == '{' ? JSMN_OBJECT : JSMN_ARRAY);
token->start = parser->pos;
parser->toksuper = parser->toknext - 1;
break;
case '}': case ']':
if (tokens == NULL)
break;
type = (c == '}' ? JSMN_OBJECT : JSMN_ARRAY);
#ifdef JSMN_PARENT_LINKS
if (parser->toknext < 1) {
return JSMN_ERROR_INVAL;
}
token = &tokens[parser->toknext - 1];
for (;;) {
if (token->start != -1 && token->end == -1) {
if (token->type != type) {
return JSMN_ERROR_INVAL;
}
token->end = parser->pos + 1;
parser->toksuper = token->parent;
break;
}
if (token->parent == -1) {
break;
}
token = &tokens[token->parent];
}
#else
for (i = parser->toknext - 1; i >= 0; i--) {
token = &tokens[i];
if (token->start != -1 && token->end == -1) {
if (token->type != type) {
return JSMN_ERROR_INVAL;
}
parser->toksuper = -1;
token->end = parser->pos + 1;
break;
}
}
/* Error if unmatched closing bracket */
if (i == -1) return JSMN_ERROR_INVAL;
for (; i >= 0; i--) {
token = &tokens[i];
if (token->start != -1 && token->end == -1) {
parser->toksuper = i;
break;
}
}
#endif
break;
case '\"':
r = jsmn_parse_string(parser, js, len, tokens, num_tokens);
if (r < 0) return r;
count++;
if (parser->toksuper != -1 && tokens != NULL)
tokens[parser->toksuper].size++;
break;
case '\t' : case '\r' : case '\n' : case ' ':
break;
case ':':
parser->toksuper = parser->toknext - 1;
break;
case ',':
if (tokens != NULL &&
tokens[parser->toksuper].type != JSMN_ARRAY &&
tokens[parser->toksuper].type != JSMN_OBJECT) {
#ifdef JSMN_PARENT_LINKS
parser->toksuper = tokens[parser->toksuper].parent;
#else
for (i = parser->toknext - 1; i >= 0; i--) {
if (tokens[i].type == JSMN_ARRAY || tokens[i].type == JSMN_OBJECT) {
if (tokens[i].start != -1 && tokens[i].end == -1) {
parser->toksuper = i;
break;
}
}
}
#endif
}
break;
#ifdef JSMN_STRICT
/* In strict mode primitives are: numbers and booleans */
case '-': case '0': case '1' : case '2': case '3' : case '4':
case '5': case '6': case '7' : case '8': case '9':
case 't': case 'f': case 'n' :
/* And they must not be keys of the object */
if (tokens != NULL) {
jsmntok_t *t = &tokens[parser->toksuper];
if (t->type == JSMN_OBJECT ||
(t->type == JSMN_STRING && t->size != 0)) {
return JSMN_ERROR_INVAL;
}
}
#else
/* In non-strict mode every unquoted value is a primitive */
default:
#endif
r = jsmn_parse_primitive(parser, js, len, tokens, num_tokens);
if (r < 0) return r;
count++;
if (parser->toksuper != -1 && tokens != NULL)
tokens[parser->toksuper].size++;
break;
#ifdef JSMN_STRICT
/* Unexpected char in strict mode */
default:
return JSMN_ERROR_INVAL;
#endif
}
}
for (i = parser->toknext - 1; i >= 0; i--) {
/* Unmatched opened object or array */
if (tokens[i].start != -1 && tokens[i].end == -1) {
return JSMN_ERROR_PART;
}
}
return count;
}
/**
* Creates a new parser based over a given buffer with an array of tokens
* available.
*/
void jsmn_init(jsmn_parser *parser) {
parser->pos = 0;
parser->toknext = 0;
parser->toksuper = -1;
}

79
externals/jsmn/jsmn.h vendored Normal file
View File

@ -0,0 +1,79 @@
#ifndef __JSMN_H_
#define __JSMN_H_
#include <stddef.h>
// APPLE2IX : define these ...
#define JSMN_PARENT_LINKS
#define JSMN_STRICT
#ifdef __cplusplus
extern "C" {
#endif
/**
* JSON type identifier. Basic types are:
* o Object
* o Array
* o String
* o Other primitive: number, boolean (true/false) or null
*/
typedef enum {
JSMN_PRIMITIVE = 0,
JSMN_OBJECT = 1,
JSMN_ARRAY = 2,
JSMN_STRING = 3
} jsmntype_t;
typedef enum {
/* Not enough tokens were provided */
JSMN_ERROR_NOMEM = -1,
/* Invalid character inside JSON string */
JSMN_ERROR_INVAL = -2,
/* The string is not a full JSON packet, more bytes expected */
JSMN_ERROR_PART = -3
} jsmnerr_t;
/**
* JSON token description.
* @param type type (object, array, string etc.)
* @param start start position in JSON data string
* @param end end position in JSON data string
*/
typedef struct {
jsmntype_t type;
int start;
int end;
int size;
#ifdef JSMN_PARENT_LINKS
int parent;
#endif
} jsmntok_t;
/**
* JSON parser. Contains an array of token blocks available. Also stores
* the string being parsed now and current position in that string
*/
typedef struct {
unsigned int pos; /* offset in the JSON string */
unsigned int toknext; /* next token to allocate */
int toksuper; /* superior token node, e.g parent object or array */
} jsmn_parser;
/**
* Create JSON parser over an array of tokens
*/
void jsmn_init(jsmn_parser *parser);
/**
* Run JSON parser. It parses a JSON data string into and array of tokens, each describing
* a single JSON object.
*/
jsmnerr_t jsmn_parse(jsmn_parser *parser, const char *js, size_t len,
jsmntok_t *tokens, unsigned int num_tokens);
#ifdef __cplusplus
}
#endif
#endif /* __JSMN_H_ */

608
externals/jsmn/jsmn_test.c vendored Normal file
View File

@ -0,0 +1,608 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
static int test_passed = 0;
static int test_failed = 0;
/* Terminate current test with error */
#define fail() return __LINE__
/* Successfull end of the test case */
#define done() return 0
/* Check single condition */
#define check(cond) do { if (!(cond)) fail(); } while (0)
/* Test runner */
static void test(int (*func)(void), const char *name) {
int r = func();
if (r == 0) {
test_passed++;
} else {
test_failed++;
printf("FAILED: %s (at line %d)\n", name, r);
}
}
#define TOKEN_EQ(t, tok_start, tok_end, tok_type) \
((t).start == tok_start \
&& (t).end == tok_end \
&& (t).type == (tok_type))
#define TOKEN_STRING(js, t, s) \
(strncmp(js+(t).start, s, (t).end - (t).start) == 0 \
&& strlen(s) == (t).end - (t).start)
#define TOKEN_PRINT(t) \
printf("start: %d, end: %d, type: %d, size: %d\n", \
(t).start, (t).end, (t).type, (t).size)
#define JSMN_STRICT
#include "jsmn.c"
int test_empty() {
const char *js;
int r;
jsmn_parser p;
jsmntok_t t[10];
js = "{}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), t, 10);
check(r >= 0);
check(t[0].type == JSMN_OBJECT);
check(t[0].start == 0 && t[0].end == 2);
js = "[]";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), t, 10);
check(r >= 0);
check(t[0].type == JSMN_ARRAY);
check(t[0].start == 0 && t[0].end == 2);
js = "{\"a\":[]}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), t, 10);
check(r >= 0);
check(t[0].type == JSMN_OBJECT && t[0].start == 0 && t[0].end == 8);
check(t[1].type == JSMN_STRING && t[1].start == 2 && t[1].end == 3);
check(t[2].type == JSMN_ARRAY && t[2].start == 5 && t[2].end == 7);
js = "[{},{}]";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), t, 10);
check(r >= 0);
check(t[0].type == JSMN_ARRAY && t[0].start == 0 && t[0].end == 7);
check(t[1].type == JSMN_OBJECT && t[1].start == 1 && t[1].end == 3);
check(t[2].type == JSMN_OBJECT && t[2].start == 4 && t[2].end == 6);
return 0;
}
int test_simple() {
const char *js;
int r;
jsmn_parser p;
jsmntok_t tokens[10];
js = "{\"a\": 0}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r >= 0);
check(TOKEN_EQ(tokens[0], 0, 8, JSMN_OBJECT));
check(TOKEN_EQ(tokens[1], 2, 3, JSMN_STRING));
check(TOKEN_EQ(tokens[2], 6, 7, JSMN_PRIMITIVE));
check(TOKEN_STRING(js, tokens[0], js));
check(TOKEN_STRING(js, tokens[1], "a"));
check(TOKEN_STRING(js, tokens[2], "0"));
jsmn_init(&p);
js = "[\"a\":{},\"b\":{}]";
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r >= 0);
jsmn_init(&p);
js = "{\n \"Day\": 26,\n \"Month\": 9,\n \"Year\": 12\n }";
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r >= 0);
return 0;
}
int test_primitive() {
#ifndef JSMN_STRICT
int r;
jsmn_parser p;
jsmntok_t tok[10];
const char *js;
js = "\"boolVar\" : true";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_STRING
&& tok[1].type == JSMN_PRIMITIVE);
check(TOKEN_STRING(js, tok[0], "boolVar"));
check(TOKEN_STRING(js, tok[1], "true"));
js = "\"boolVar\" : false";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_STRING
&& tok[1].type == JSMN_PRIMITIVE);
check(TOKEN_STRING(js, tok[0], "boolVar"));
check(TOKEN_STRING(js, tok[1], "false"));
js = "\"intVar\" : 12345";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_STRING
&& tok[1].type == JSMN_PRIMITIVE);
check(TOKEN_STRING(js, tok[0], "intVar"));
check(TOKEN_STRING(js, tok[1], "12345"));
js = "\"floatVar\" : 12.345";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_STRING
&& tok[1].type == JSMN_PRIMITIVE);
check(TOKEN_STRING(js, tok[0], "floatVar"));
check(TOKEN_STRING(js, tok[1], "12.345"));
js = "\"nullVar\" : null";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_STRING
&& tok[1].type == JSMN_PRIMITIVE);
check(TOKEN_STRING(js, tok[0], "nullVar"));
check(TOKEN_STRING(js, tok[1], "null"));
#endif
return 0;
}
int test_string() {
int r;
jsmn_parser p;
jsmntok_t tok[10];
const char *js;
js = "\"strVar\" : \"hello world\"";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_STRING
&& tok[1].type == JSMN_STRING);
check(TOKEN_STRING(js, tok[0], "strVar"));
check(TOKEN_STRING(js, tok[1], "hello world"));
js = "\"strVar\" : \"escapes: \\/\\r\\n\\t\\b\\f\\\"\\\\\"";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_STRING
&& tok[1].type == JSMN_STRING);
check(TOKEN_STRING(js, tok[0], "strVar"));
check(TOKEN_STRING(js, tok[1], "escapes: \\/\\r\\n\\t\\b\\f\\\"\\\\"));
js = "\"strVar\" : \"\"";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_STRING
&& tok[1].type == JSMN_STRING);
check(TOKEN_STRING(js, tok[0], "strVar"));
check(TOKEN_STRING(js, tok[1], ""));
return 0;
}
int test_partial_string() {
int r;
jsmn_parser p;
jsmntok_t tok[10];
const char *js;
jsmn_init(&p);
js = "\"x\": \"va";
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r == JSMN_ERROR_PART && tok[0].type == JSMN_STRING);
check(TOKEN_STRING(js, tok[0], "x"));
check(p.toknext == 1);
jsmn_init(&p);
char js_slash[9] = "\"x\": \"va\\";
r = jsmn_parse(&p, js_slash, sizeof(js_slash), tok, 10);
check(r == JSMN_ERROR_PART);
jsmn_init(&p);
char js_unicode[10] = "\"x\": \"va\\u";
r = jsmn_parse(&p, js_unicode, sizeof(js_unicode), tok, 10);
check(r == JSMN_ERROR_PART);
js = "\"x\": \"valu";
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r == JSMN_ERROR_PART && tok[0].type == JSMN_STRING);
check(TOKEN_STRING(js, tok[0], "x"));
check(p.toknext == 1);
js = "\"x\": \"value\"";
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_STRING
&& tok[1].type == JSMN_STRING);
check(TOKEN_STRING(js, tok[0], "x"));
check(TOKEN_STRING(js, tok[1], "value"));
js = "\"x\": \"value\", \"y\": \"value y\"";
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_STRING
&& tok[1].type == JSMN_STRING && tok[2].type == JSMN_STRING
&& tok[3].type == JSMN_STRING);
check(TOKEN_STRING(js, tok[0], "x"));
check(TOKEN_STRING(js, tok[1], "value"));
check(TOKEN_STRING(js, tok[2], "y"));
check(TOKEN_STRING(js, tok[3], "value y"));
return 0;
}
int test_unquoted_keys() {
#ifndef JSMN_STRICT
int r;
jsmn_parser p;
jsmntok_t tok[10];
const char *js;
jsmn_init(&p);
js = "key1: \"value\"\nkey2 : 123";
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_PRIMITIVE
&& tok[1].type == JSMN_STRING && tok[2].type == JSMN_PRIMITIVE
&& tok[3].type == JSMN_PRIMITIVE);
check(TOKEN_STRING(js, tok[0], "key1"));
check(TOKEN_STRING(js, tok[1], "value"));
check(TOKEN_STRING(js, tok[2], "key2"));
check(TOKEN_STRING(js, tok[3], "123"));
#endif
return 0;
}
int test_partial_array() {
int r;
jsmn_parser p;
jsmntok_t tok[10];
const char *js;
jsmn_init(&p);
js = " [ 1, true, ";
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r == JSMN_ERROR_PART && tok[0].type == JSMN_ARRAY
&& tok[1].type == JSMN_PRIMITIVE && tok[2].type == JSMN_PRIMITIVE);
js = " [ 1, true, [123, \"hello";
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r == JSMN_ERROR_PART && tok[0].type == JSMN_ARRAY
&& tok[1].type == JSMN_PRIMITIVE && tok[2].type == JSMN_PRIMITIVE
&& tok[3].type == JSMN_ARRAY && tok[4].type == JSMN_PRIMITIVE);
js = " [ 1, true, [123, \"hello\"]";
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r == JSMN_ERROR_PART && tok[0].type == JSMN_ARRAY
&& tok[1].type == JSMN_PRIMITIVE && tok[2].type == JSMN_PRIMITIVE
&& tok[3].type == JSMN_ARRAY && tok[4].type == JSMN_PRIMITIVE
&& tok[5].type == JSMN_STRING);
/* check child nodes of the 2nd array */
check(tok[3].size == 2);
js = " [ 1, true, [123, \"hello\"]]";
r = jsmn_parse(&p, js, strlen(js), tok, 10);
check(r >= 0 && tok[0].type == JSMN_ARRAY
&& tok[1].type == JSMN_PRIMITIVE && tok[2].type == JSMN_PRIMITIVE
&& tok[3].type == JSMN_ARRAY && tok[4].type == JSMN_PRIMITIVE
&& tok[5].type == JSMN_STRING);
check(tok[3].size == 2);
check(tok[0].size == 3);
return 0;
}
int test_array_nomem() {
int i;
int r;
jsmn_parser p;
jsmntok_t toksmall[10], toklarge[10];
const char *js;
js = " [ 1, true, [123, \"hello\"]]";
for (i = 0; i < 6; i++) {
jsmn_init(&p);
memset(toksmall, 0, sizeof(toksmall));
memset(toklarge, 0, sizeof(toklarge));
r = jsmn_parse(&p, js, strlen(js), toksmall, i);
check(r == JSMN_ERROR_NOMEM);
memcpy(toklarge, toksmall, sizeof(toksmall));
r = jsmn_parse(&p, js, strlen(js), toklarge, 10);
check(r >= 0);
check(toklarge[0].type == JSMN_ARRAY && toklarge[0].size == 3);
check(toklarge[3].type == JSMN_ARRAY && toklarge[3].size == 2);
}
return 0;
}
int test_objects_arrays() {
int r;
jsmn_parser p;
jsmntok_t tokens[10];
const char *js;
js = "[10}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == JSMN_ERROR_INVAL);
js = "[10]";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r >= 0);
js = "{\"a\": 1]";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == JSMN_ERROR_INVAL);
js = "{\"a\": 1}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r >= 0);
return 0;
}
int test_issue_22() {
int r;
jsmn_parser p;
jsmntok_t tokens[128];
const char *js;
js = "{ \"height\":10, \"layers\":[ { \"data\":[6,6], \"height\":10, "
"\"name\":\"Calque de Tile 1\", \"opacity\":1, \"type\":\"tilelayer\", "
"\"visible\":true, \"width\":10, \"x\":0, \"y\":0 }], "
"\"orientation\":\"orthogonal\", \"properties\": { }, \"tileheight\":32, "
"\"tilesets\":[ { \"firstgid\":1, \"image\":\"..\\/images\\/tiles.png\", "
"\"imageheight\":64, \"imagewidth\":160, \"margin\":0, \"name\":\"Tiles\", "
"\"properties\":{}, \"spacing\":0, \"tileheight\":32, \"tilewidth\":32 }], "
"\"tilewidth\":32, \"version\":1, \"width\":10 }";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 128);
check(r >= 0);
#if 0
for (i = 1; tokens[i].end < tokens[0].end; i++) {
if (tokens[i].type == JSMN_STRING || tokens[i].type == JSMN_PRIMITIVE) {
printf("%.*s\n", tokens[i].end - tokens[i].start, js + tokens[i].start);
} else if (tokens[i].type == JSMN_ARRAY) {
printf("[%d elems]\n", tokens[i].size);
} else if (tokens[i].type == JSMN_OBJECT) {
printf("{%d elems}\n", tokens[i].size);
} else {
TOKEN_PRINT(tokens[i]);
}
}
#endif
return 0;
}
int test_unicode_characters() {
jsmn_parser p;
jsmntok_t tokens[10];
const char *js;
int r;
js = "{\"a\":\"\\uAbcD\"}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r >= 0);
js = "{\"a\":\"str\\u0000\"}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r >= 0);
js = "{\"a\":\"\\uFFFFstr\"}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r >= 0);
js = "{\"a\":\"str\\uFFGFstr\"}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == JSMN_ERROR_INVAL);
js = "{\"a\":\"str\\u@FfF\"}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == JSMN_ERROR_INVAL);
js = "{\"a\":[\"\\u028\"]}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == JSMN_ERROR_INVAL);
js = "{\"a\":[\"\\u0280\"]}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r >= 0);
return 0;
}
int test_input_length() {
const char *js;
int r;
jsmn_parser p;
jsmntok_t tokens[10];
js = "{\"a\": 0}garbage";
jsmn_init(&p);
r = jsmn_parse(&p, js, 8, tokens, 10);
check(r == 3);
check(TOKEN_STRING(js, tokens[0], "{\"a\": 0}"));
check(TOKEN_STRING(js, tokens[1], "a"));
check(TOKEN_STRING(js, tokens[2], "0"));
return 0;
}
int test_count() {
jsmn_parser p;
const char *js;
js = "{}";
jsmn_init(&p);
check(jsmn_parse(&p, js, strlen(js), NULL, 0) == 1);
js = "[]";
jsmn_init(&p);
check(jsmn_parse(&p, js, strlen(js), NULL, 0) == 1);
js = "[[]]";
jsmn_init(&p);
check(jsmn_parse(&p, js, strlen(js), NULL, 0) == 2);
js = "[[], []]";
jsmn_init(&p);
check(jsmn_parse(&p, js, strlen(js), NULL, 0) == 3);
js = "[[], []]";
jsmn_init(&p);
check(jsmn_parse(&p, js, strlen(js), NULL, 0) == 3);
js = "[[], [[]], [[], []]]";
jsmn_init(&p);
check(jsmn_parse(&p, js, strlen(js), NULL, 0) == 7);
js = "[\"a\", [[], []]]";
jsmn_init(&p);
check(jsmn_parse(&p, js, strlen(js), NULL, 0) == 5);
js = "[[], \"[], [[]]\", [[]]]";
jsmn_init(&p);
check(jsmn_parse(&p, js, strlen(js), NULL, 0) == 5);
js = "[1, 2, 3]";
jsmn_init(&p);
check(jsmn_parse(&p, js, strlen(js), NULL, 0) == 4);
js = "[1, 2, [3, \"a\"], null]";
jsmn_init(&p);
check(jsmn_parse(&p, js, strlen(js), NULL, 0) == 7);
return 0;
}
int test_keyvalue() {
const char *js;
int r;
jsmn_parser p;
jsmntok_t tokens[10];
js = "{\"a\": 0, \"b\": \"c\"}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == 5);
check(tokens[0].size == 2); /* two keys */
check(tokens[1].size == 1 && tokens[3].size == 1); /* one value per key */
check(tokens[2].size == 0 && tokens[4].size == 0); /* values have zero size */
js = "{\"a\"\n0}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == JSMN_ERROR_INVAL);
js = "{\"a\", 0}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == JSMN_ERROR_INVAL);
js = "{\"a\": {2}}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == JSMN_ERROR_INVAL);
js = "{\"a\": {2: 3}}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == JSMN_ERROR_INVAL);
js = "{\"a\": {\"a\": 2 3}}";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == JSMN_ERROR_INVAL);
return 0;
}
/** A huge redefinition of everything to include jsmn in non-script mode */
#define jsmn_init jsmn_init_nonstrict
#define jsmn_parse jsmn_parse_nonstrict
#define jsmn_parser jsmn_parser_nonstrict
#define jsmn_alloc_token jsmn_alloc_token_nonstrict
#define jsmn_fill_token jsmn_fill_token_nonstrict
#define jsmn_parse_primitive jsmn_parse_primitive_nonstrict
#define jsmn_parse_string jsmn_parse_string_nonstrict
#define jsmntype_t jsmntype_nonstrict_t
#define jsmnerr_t jsmnerr_nonstrict_t
#define jsmntok_t jsmntok_nonstrict_t
#define JSMN_PRIMITIVE JSMN_PRIMITIVE_NONSTRICT
#define JSMN_OBJECT JSMN_OBJECT_NONSTRICT
#define JSMN_ARRAY JSMN_ARRAY_NONSTRICT
#define JSMN_STRING JSMN_STRING_NONSTRICT
#define JSMN_ERROR_NOMEM JSMN_ERROR_NOMEM_NONSTRICT
#define JSMN_ERROR_INVAL JSMN_ERROR_INVAL_NONSTRICT
#define JSMN_ERROR_PART JSMN_ERROR_PART_NONSTRICT
#undef __JSMN_H_
#undef JSMN_STRICT
#include "jsmn.c"
int test_nonstrict() {
const char *js;
int r;
jsmn_parser p;
jsmntok_t tokens[10];
js = "a: 0garbage";
jsmn_init(&p);
r = jsmn_parse(&p, js, 4, tokens, 10);
check(r == 2);
check(TOKEN_STRING(js, tokens[0], "a"));
check(TOKEN_STRING(js, tokens[1], "0"));
js = "Day : 26\nMonth : Sep\n\nYear: 12";
jsmn_init(&p);
r = jsmn_parse(&p, js, strlen(js), tokens, 10);
check(r == 6);
return 0;
}
int main() {
test(test_empty, "general test for a empty JSON objects/arrays");
test(test_simple, "general test for a simple JSON string");
test(test_primitive, "test primitive JSON data types");
test(test_string, "test string JSON data types");
test(test_partial_string, "test partial JSON string parsing");
test(test_partial_array, "test partial array reading");
test(test_array_nomem, "test array reading with a smaller number of tokens");
test(test_unquoted_keys, "test unquoted keys (like in JavaScript)");
test(test_objects_arrays, "test objects and arrays");
test(test_unicode_characters, "test unicode characters");
test(test_input_length, "test strings that are not null-terminated");
test(test_issue_22, "test issue #22");
test(test_count, "test tokens count estimation");
test(test_nonstrict, "test for non-strict mode");
test(test_keyvalue, "test for keys/values");
printf("\nPASSED: %d\nFAILED: %d\n", test_passed, test_failed);
return 0;
}

209
src/json_parse.c Normal file
View File

@ -0,0 +1,209 @@
/*
* Apple // emulator for *nix
*
* This software package is subject to the GNU General Public License
* version 2 or later (your choice) as published by the Free Software
* Foundation.
*
* THERE ARE NO WARRANTIES WHATSOEVER.
*
*/
#include "json_parse.h"
#define JSON_LENGTH 16
#define DEFAULT_NUMTOK 16
int json_createFromFile(const char *filePath, INOUT JSON_s *parsedData) {
int fd = -1;
jsmnerr_t errCount = JSMN_ERROR_NOMEM;
ssize_t jsonIdx = 0;
ssize_t jsonLen = 0;
char *jsonString = NULL;
jsmntok_t *jsonTokens = NULL;
do {
if (!parsedData) {
break;
}
parsedData->jsonString = NULL;
parsedData->jsonTokens = NULL;
TEMP_FAILURE_RETRY(fd = open(filePath, O_RDONLY));
if (fd < 0) {
ERRLOG("Error opening file : %s", strerror(errno));
break;
}
jsonLen = JSON_LENGTH*2;
jsonString = malloc(jsonLen);
if (jsonString == NULL) {
ERRLOG("WHOA : %s", strerror(errno));
break;
}
ssize_t bytesRead = 0;
do {
TEMP_FAILURE_RETRY(bytesRead = read(fd, jsonString+jsonIdx, JSON_LENGTH));
if (bytesRead < 0) {
ERRLOG("Error reading file : %s", strerror(errno));
break;
}
if (bytesRead) {
jsonIdx += bytesRead;
if (jsonLen - jsonIdx < JSON_LENGTH) {
//LOG("reallocating json string ...");
jsonLen <<= 1;
char *newString = realloc(jsonString, jsonLen);
if (!newString) {
ERRLOG("WHOA2 : %s", strerror(errno));
bytesRead = -1;
break;
}
jsonString = newString;
}
}
} while (bytesRead);
if (bytesRead < 0) {
break;
}
jsonLen = jsonIdx;
TEMP_FAILURE_RETRY(close(fd));
fd = 0;
// now parse the string
jsmn_parser parser = { 0 };
unsigned int numTokens = DEFAULT_NUMTOK;
do {
if (!jsonTokens) {
jsonTokens = calloc(numTokens, sizeof(jsmntok_t));
if (!jsonTokens) {
ERRLOG("WHOA3 : %s", strerror(errno));
break;
}
} else {
//LOG("reallocating json tokens ...");
numTokens <<= 1;
jsmntok_t *newTokens = realloc(jsonTokens, numTokens * sizeof(jsmntok_t));
memset(newTokens, '\0', numTokens * sizeof(jsmntok_t));
if (!newTokens) {
ERRLOG("WHOA4 : %s", strerror(errno));
break;
}
jsonTokens = newTokens;
}
jsmn_init(&parser);
errCount = jsmn_parse(&parser, jsonString, jsonLen, jsonTokens, numTokens-(numTokens/2));
} while (errCount == JSMN_ERROR_NOMEM);
if (errCount == JSMN_ERROR_NOMEM) {
break;
}
if (errCount < 0) {
ERRLOG("%s", "OOPS error parsing JSON : ");
switch (errCount) {
case JSMN_ERROR_NOMEM:
assert(0 && "should not happen");
break;
case JSMN_ERROR_INVAL:
ERRLOG("%s", "Invalid character inside JSON string");
break;
case JSMN_ERROR_PART:
ERRLOG("%s", "String is not a complete JSON packet, moar bytes expected");
break;
default:
ERRLOG("UNKNOWN errCount : %d", errCount);
break;
}
break;
}
parsedData->jsonString = jsonString;
parsedData->jsonTokens = jsonTokens;
} while (0);
if (fd > 0) {
TEMP_FAILURE_RETRY(close(fd));
fd = 0;
}
if (errCount < 0) {
if (jsonString) {
FREE(jsonString);
}
if (jsonTokens) {
FREE(jsonTokens);
}
}
return errCount;
}
void json_destroy(JSON_s *parsedData) {
FREE(parsedData->jsonString);
FREE(parsedData->jsonTokens);
}
#if TEST_JSON
bool do_logging = true;
FILE *error_log = NULL;
// Runs jsmn against an arbitrary a test_file.json
int main(int argc, char **argv) {
error_log = stderr;
if (argc < 2) {
fprintf(stderr, "Please specify a JSON file to parse on CLI, e.g. : %s path/to/some_file.json\n", argv[0]);
exit(1);
}
JSON_s parsedData = { 0 };
int tokCount = json_createFromFile(argv[1], &parsedData);
if (tokCount < 0) {
return 1;
}
fprintf(stderr, "TOK COUNT:%d\n", tokCount);
for (int i=0; i<tokCount; i++) {
fprintf(stderr, "%s", "type:");
switch(parsedData.jsonTokens[i].type) {
case JSMN_PRIMITIVE:
fprintf(stderr, "%s", "PRIMITIVE ");
break;
case JSMN_OBJECT:
fprintf(stderr, "%s", "OBJECT ");
break;
case JSMN_ARRAY:
fprintf(stderr, "%s", "ARRAY ");
break;
case JSMN_STRING:
fprintf(stderr, "%s", "STRING ");
break;
default:
fprintf(stderr, "%s", "UNKNOWN ");
break;
}
int start = parsedData.jsonTokens[i].start;
int end = parsedData.jsonTokens[i].end;
fprintf(stderr, "start:%6d end:%6d size:%6d parent:%3d", start, end, parsedData.jsonTokens[i].size, parsedData.jsonTokens[i].parent);
if (parsedData.jsonTokens[i].type == JSMN_STRING) {
char lastChar = parsedData.jsonString[end];
parsedData.jsonString[end] = '\0';
fprintf(stderr, " VALUE:%s", &parsedData.jsonString[start]);
parsedData.jsonString[end] = lastChar;
}
fprintf(stderr, "%s", "\n");
}
return 0;
}
#endif

25
src/json_parse.h Normal file
View File

@ -0,0 +1,25 @@
/*
* Apple // emulator for *nix
*
* This software package is subject to the GNU General Public License
* version 2 or later (your choice) as published by the Free Software
* Foundation.
*
* THERE ARE NO WARRANTIES WHATSOEVER.
*
*/
#include "common.h"
#include "../externals/jsmn/jsmn.h"
typedef struct JSON_s {
char *jsonString;
jsmntok_t *jsonTokens;
} JSON_s;
// parses file into string and tokens. returns positive token count or negative jsmnerr_t error code.
int json_createFromFile(const char *filePath, INOUT JSON_s *parsedData);
// destroys internal allocated memory (if any)
void json_destroy(JSON_s *parsedData);