2 Lexer Contract and The Don't-Stop Structure Type

▼ |  | Syntax Color: Utilities

1 | Parenthesis Matching
2 | Lexer Contract and The Don’t-Stop Structure Type
3 | Racket Lexer
4 | Default Lexer
5 | Module Lexer
6 | Scribble Lexer
7 | Splay Tree for Tokenization
8 | Interface for Clients of Syntax Color Output
9 | Racket S-Expression Indentation and Navigation

On this page:

lexer/ c
lexer*/ c
lexer*/ c-without-random-testing
dont-stop
check-colorer-results-match-port-before-and-afters

Racket
top contents ← prev up next →

## 2 Lexer Contract and The Don’t-Stop Structure Type🔗ℹ

(require syntax-color/lexer-contract)
| package: syntax-color-lib

> value lexer/c : contract?

Checks to be sure a lexing function is well-behaved, constrained to functions where the second return value is a symbol. For more details, see start-colorer in color:text<%>.

> value lexer*/c : contract?

Checks to be sure a lexing function is well-behaved. For more details, see start-colorer in color:text<%>.

It also supplies a few random arguments to the lexer and checks the results, using option/c’s #:tester argument.

Added in version 1.2 of package syntax-color-lib.

> value lexer*/c-without-random-testing : contract?

The same contract as lexer*/c, except without the random testing.

> struct (struct dont-stop (val))
> val : any/c

A structure type used to indicate to the lexer that it should not allow itself to be interrupted. For more details, see start-colorer in color:text<%>.

> > procedure
> > (check-colorer-results-match-port-before-and-afters |
> > |  | who |
> > |  | type |
> > |  | pos-before |
> > |  | new-token-start |
> > |  | new-token-end |
> > |  | pos-after) |
> → void?
> who : symbol?
> type : any/c
> pos-before : (or/c exact-positive-integer? #f)
> new-token-start : (or/c exact-positive-integer? #f)
> new-token-end : (or/c exact-positive-integer? #f)
> pos-after : (or/c exact-positive-integer? #f)

Checks that the results of a colorer make sense with respect to the positions of the port, before and after the lexer is called.

The pos-before argument is expected to be the third result of port-next-location before a lexer is called and the pos-after argument is expected to to be the third result of port-next-location after the lexer is called. The type, token-start, and token-end arguments should be the corresponding results from the colorer (c.f. start-colorer).

This function raises an error unless the following boolean expression is true:

> (or (equal? type 'eof)
> (and (= pos-before new-token-start)
> (< new-token-start new-token-end)
> (= new-token-end pos-after)))

but it checks the individual parts of the expression to raise a more meaningful error message when some part is not true.

The who argument is used to start the error message.

Added in version 1.4 of package syntax-color-lib.

top contents ← prev up next →