(text xml read)

Overview

Reading XML: as signals, as a fold, as a tree of your own, or as a Document.

(import (text xml) (text xml read))

(parse-xml "<feed><entry id='1'>hello</entry></feed>")   ; (Ok document)
(parse-xml (xml-file "feed.xml") #:whitespace 'drop)
(xml-fold source 0 #:new-level (fun (name attrs pos n) (+ n 1)))   ; the number of elements

Four layers, each built on the one below: signals, a pull stream as OCaml’s Xmlm has it; xml-fold, Kiselyov’s SSAX fold; xml-tree-fold, a tree of your own type; and parse-xml, a Document, with xml-for-each to read one element at a time. Only the signal layer calls XmlReader.

Every reader takes a source, anything with an XmlSource implementation, and the same options.

Options.

keyword	default	what it does
`#:dtd`	`'prohibit`	`'ignore` skips a DOCTYPE; `'parse` reads one
`#:max-depth`	`512`	more nesting than this is an error
`#:max-characters`	`0`	a limit on the document’s length; `0` is none
`#:whitespace`	`'keep`	`'drop` leaves out text that is only whitespace
`#:comments?`	`#f`	whether comments are read (signals, `parse-xml`, `xml-for-each`)

The defaults are the safe ones. 'prohibit refuses a document with a DOCTYPE. 'ignore skips it, and then XmlReader does not report it at all, so the document has no doctype, and an entity it declared is an error. 'parse reads it, never resolves anything external (an external DTD or entity is an error) and expands at most 10 000 000 characters of entities, which stops a “billion laughs” document quickly. So an XHTML document, which names an external DTD, reads only under 'ignore.

'drop drops text that is only spaces, tabs and line ends. That includes the space in x y, which then reads xy, and whitespace inside xml:space="preserve".

The folds take a #:comment handler instead of #:comments?.

A #:max-depth under 1 or a negative #:max-characters raises an ArgumentException: that is a programming error, not the document’s.

Errors. What is wrong with the document is an XmlError, answered in a Result. An exception your own handler raises is never an XmlError: it passes through as it was, an XmlException of your own included.

A reader streams, so one that fails at a node has already run its handlers for every node before it.

Reference

Types

`DtdMode`

union

What a reader does with a DOCTYPE: #:dtd’s value.

DtdProhibit: 'prohibit, the default: a DOCTYPE is an error.
DtdIgnore: 'ignore: it is skipped, and not reported, so the document has no doctype and an entity it declared is an error.
DtdParse: 'parse: it is read, nothing external is resolved, and at most 10 000 000 characters of entities are expanded.

`WhitespaceMode`

union

Whether a reader keeps text that is only whitespace: #:whitespace’s value.

KeepWhitespace: 'keep, the default: every character of text is kept.
DropWhitespace: 'drop: text that is only spaces, tabs and line ends is left out.

'drop includes the space in x y, which then reads xy, and whitespace inside xml:space="preserve".

`XmlError`

record

What is wrong with a document, and where.

kind: Which kind of error it is.
message: What .NET or the reader says.
line: The line, from 1; 0 where there is no position.
column: The column, from 1; 0 where there is no position.

line and column are 0 where .NET gives no position, which it does not for a DOCTYPE refused or a limit reached. ->str writes all of it on one line: malformed at line 1, column 16: ....

`XmlErrorKind`

union

Which kind of error an XmlError is.

Malformed: The document is not well-formed XML.
DtdProhibited: A DOCTYPE under 'prohibit, or an external DTD or entity under 'parse.
TooDeep: More nesting than #:max-depth.
LimitExceeded: More than #:max-characters, or the entity limit under 'parse.

`XmlInput`

union

What a source is read as: what xml-input answers.

Opaque. A type of your own gets one by handing xml-input something that already has an XmlSource implementation.

`XmlSource`

trait

What can be read as XML.

%s: The source's type.

(impl (XmlSource Upload)
  (defun (xml-input u) (xml-input (upload-stream u))))

These have an implementation:

`string`	the XML itself
`TextInputPort`	read as it is; left open
`Stream`	bytes, whose encoding the reader works out from a byte order mark or the XML declaration, else UTF-8; left open
`ByteInputPort`	the same, through the port, so nothing it buffered is lost
`(xml-file path)`	opened through the `FS` effect and closed again

A port or stream you hand over stays open: whoever opens a resource closes it.

`XmlFile`

record

A path to read XML from, as opposed to a string of XML.

`XmlSignal`

union

One step through a document, as xml-next! answers it.

XmlStart: An element begins: its name, attributes and position.
XmlEnd: The element last begun ends.
XmlData: Text.
XmlComment: A comment, with #:comments? #t.
XmlPI: A processing instruction: its target and data.
XmlDoctype: The doctype, under 'parse.

An empty element,  , is an XmlStart and an XmlEnd like any other.
Adjacent text, CDATA and whitespace are one XmlData, even across a comment that is not being read. a<![CDATA[b]]>c is "abc".
Whitespace outside the root element is not data.

`XmlSignals`

record

A document being read, signal by signal.

Functions

`xml-input`

function

(: xml-input (-> %s XmlInput))

A source as the reader takes it.

source: The source.
returns: What it reads as.

`xml-file`

function

(: xml-file (-> string XmlFile))
(xml-file path)

A file to read, as a source.

path: The file's path.
returns: A source the reader opens, through the FS effect, and closes again.

(parse-xml (xml-file "feed.xml"))

Nothing is opened until a reader reads it. A missing file then raises, as file-read-text does; it is not an XmlError.

`call-with-xml-signals`

function

(: call-with-xml-signals (-> %s (-> XmlSignals %a) (#:dtd DtdMode) (#:max-depth int) (#:max-characters long) (#:whitespace WhitespaceMode) (#:comments? bool) %a))
(call-with-xml-signals source proc
                              #:dtd 'prohibit #:max-depth 512 #:max-characters 0L
                              #:whitespace 'keep #:comments? #f)

Calls a function with a pull stream of a document’s signals.

source: What to read.
proc: Called with the signals, which are closed when it returns.
#:dtd: What to do with a DOCTYPE.
#:max-depth: How deep elements may nest.
#:max-characters: A limit on the document's length; 0 is none.
#:whitespace: Whether to keep text that is only whitespace.
#:comments?: Whether comments are read.
returns: What proc answers.
raises ArgumentException: When #:max-depth is less than 1 or #:max-characters is negative.
raises System.IO.IOException: When the source is an xml-file that cannot be opened.

(call-with-xml-signals source
  (fun (signals)
    (match (xml-next! signals)
      ((Ok (Some (XmlStart name attrs pos))) ...)
      ((Ok (Some XmlEnd)) ...)
      ((Ok (Some (XmlData text))) ...)
      ((Ok None) ...)                     ; the end of the document
      ((Err e) ...)
      (_ ...))))

The bottom layer, as OCaml’s Xmlm has it. The signals are for proc’s use only. An error the reader meets before the first signal is answered by the first xml-next!.

`xml-next!`

function

(: xml-next! (-> XmlSignals (Result XmlError (Option XmlSignal))))
(xml-next! s)

The next signal of a document.

s: The signals.
returns: Some signal, None at the end of the document, or the error. After an error, the same error again.

`xml-skip-element!`

function

(: xml-skip-element! (-> XmlSignals (Result XmlError Unit)))
(xml-skip-element! s)

Skips the rest of the element just begun.

s: The signals, right after an XmlStart.
returns: Ok when the rest of the element, its XmlEnd included, has been skipped, or the error met on the way.
raises InvalidOperationException: When the last signal read was not an XmlStart.

The skipped part is not checked against #:max-depth, though #:max-characters still counts it.

`xml-fold`

function

(: xml-fold (-> %s %seed (#:new-level (-> XmlName XmlAttrs (Option XmlPos) %seed %seed)) (#:finish (-> XmlName XmlAttrs (Option XmlPos) %seed %seed %seed)) (#:text (-> string %seed %seed)) (#:comment (-> string %seed %seed)) (#:pi (-> string string %seed %seed)) (#:doctype (-> Doctype %seed %seed)) (#:dtd DtdMode) (#:max-depth int) (#:max-characters long) (#:whitespace WhitespaceMode) (Result XmlError %seed)))
(xml-fold source seed
                 #:new-level (fun (name attrs pos seed) seed)
                 #:finish (fun (name attrs pos parent seed) seed)
                 #:text (fun (s seed) seed)
                 #:comment (fun (s seed) seed)
                 #:pi (fun (target data seed) seed)
                 #:doctype (fun (dt seed) seed)
                 #:dtd 'prohibit #:max-depth 512 #:max-characters 0L #:whitespace 'keep)

Kiselyov’s SSAX fold: a seed threaded through a document.

source: What to read.
seed: The seed to begin with.
#:new-level: Going into an element: its name, attributes, position and the seed; answers the element's own seed.
#:finish: Coming out: name, attributes, position, the seed from before the element and the element's final seed; answers the seed to carry on with.
#:text: A piece of text and the seed.
#:comment: A comment and the seed.
#:pi: A processing instruction's target and data, and the seed.
#:doctype: The doctype, under 'parse, and the seed.
#:dtd: What to do with a DOCTYPE.
#:max-depth: How deep elements may nest.
#:max-characters: A limit on the document's length; 0 is none.
#:whitespace: Whether to keep text that is only whitespace.
returns: The final seed, or the error.
raises ArgumentException: When #:max-depth is less than 1 or #:max-characters is negative.
raises System.IO.IOException: When the source is an xml-file that cannot be opened.

(xml-fold source 0 #:new-level (fun (name attrs pos n) (+ n 1)))     ; the number of elements

;; (Ok "(a(b x)(c))")
(xml-fold "<a><b>x</b><c/></a>" ""
          #:new-level (fun (name attrs pos seed) "")
          #:finish (fun (name attrs pos parent seed) (str parent "(" (xml-name-local name) seed ")"))
          #:text (fun (s seed) (str seed " " s)))

Every handler passes the seed through unless it is given. The seed is an argument rather than a keyword, since a keyword needs a default and a seed of any type has none.

The fold is a loop over the signals, with the open elements in a list, so a flat document of a million elements and one nested 100 000 deep are the same to it.

`xml-tree-fold`

function

(: xml-tree-fold (-> %s (-> XmlName XmlAttrs (Option XmlPos) (List %a) %a) (-> string %a) (#:comment (-> string (Option %a))) (#:pi (-> string string (Option %a))) (#:dtd DtdMode) (#:max-depth int) (#:max-characters long) (#:whitespace WhitespaceMode) (Result XmlError %a)))
(xml-tree-fold source element text
                      #:comment (fun (s) None)
                      #:pi (fun (target data) None)
                      #:dtd 'prohibit #:max-depth 512 #:max-characters 0L #:whitespace 'keep)

A tree of your own type, as Xmlm’s input_tree builds one.

source: What to read.
element: Called with an element's name, attributes, position and its children already folded.
text: Called with a piece of text.
#:comment: Called with a comment; None leaves it out, which is what the default does.
#:pi: Called with a processing instruction's target and data; None leaves it out, which is what the default does.
#:dtd: What to do with a DOCTYPE.
#:max-depth: How deep elements may nest.
#:max-characters: A limit on the document's length; 0 is none.
#:whitespace: Whether to keep text that is only whitespace.
returns: The root element's tree, or the error. What is outside the root is not folded.
raises ArgumentException: When #:max-depth is less than 1 or #:max-characters is negative.
raises System.IO.IOException: When the source is an xml-file that cannot be opened.

(xml-tree-fold source
               (fun (name attrs pos kids) (Branch (xml-name-local name) kids))
               (fun (s) (Leaf s))
               #:comment (fun (s) (Some (Note s))))

`parse-xml`

function

(: parse-xml (-> %s (#:dtd DtdMode) (#:max-depth int) (#:max-characters long) (#:whitespace WhitespaceMode) (#:comments? bool) (Result XmlError Document)))
(parse-xml source
                  #:dtd 'prohibit #:max-depth 512 #:max-characters 0L
                  #:whitespace 'keep #:comments? #f)

Reads a whole document.

source: What to read.
#:dtd: What to do with a DOCTYPE.
#:max-depth: How deep elements may nest.
#:max-characters: A limit on the document's length; 0 is none.
#:whitespace: Whether to keep text that is only whitespace.
#:comments?: Whether comments are kept.
returns: The Document, or the error.
raises ArgumentException: When #:max-depth is less than 1 or #:max-characters is negative.
raises System.IO.IOException: When the source is an xml-file that cannot be opened.

(match (parse-xml "<feed><entry id='1'>hello</entry></feed>")
  ((Ok doc) (document-root doc))
  ((Err e) (println e)))

The document is built with the node constructors, so it keeps their rules. Processing instructions are always kept, comments with #:comments? #t, in the prolog and epilog as in the tree.

`xml-for-each`

function

(: xml-for-each (-> %s XmlName (-> Node void) (#:dtd DtdMode) (#:max-depth int) (#:max-characters long) (#:whitespace WhitespaceMode) (#:comments? bool) (Result XmlError Unit)))
(xml-for-each source name f
                     #:dtd 'prohibit #:max-depth 512 #:max-characters 0L
                     #:whitespace 'keep #:comments? #f)

Calls a function with each outermost element of a name, one at a time.

source: What to read.
name: The name of the elements to hand over.
f: Called with each one, as a Node.
#:dtd: What to do with a DOCTYPE.
#:max-depth: How deep elements may nest.
#:max-characters: A limit on the document's length; 0 is none.
#:whitespace: Whether to keep text that is only whitespace.
#:comments?: Whether comments are kept in the elements.
returns: Ok when the document has been read, or the error.
raises ArgumentException: When #:max-depth is less than 1 or #:max-characters is negative.
raises System.IO.IOException: When the source is an xml-file that cannot be opened.

(xml-for-each (xml-file "feed.xml") (xml-name "entry" #:ns atom)
              (fun (entry) (store! (text-content entry))))

Burst mode: each element is built, handed to f and dropped, so what is held at a time is one such element, never the whole document. One inside another is part of the outer one, and is not handed over by itself.