pymaiml.serialization

serialization

pymaiml.serialization

Converts maiml_domain object trees into real MaiML (JIS K 0200 / MaiML-Schema-1_0) XML text/files.

maiml_domain deliberately has no XML (de)serialization of its own -- see its README. This module is the "graduated" version of the throwaway converter that MaiML-Domain/tests/build_sample_maiml.py used to validate the domain model during development: the same recursive property/content handling, but now generalized (via pymaiml._xsi_registry) to cover every property/content type maiml_domain defines, and extended to cover every structural element (document/protocol/data/eventLog/pnml), not just the narrow slice one sample file happened to exercise.

Element ordering in every writer function below follows MaiML-Schema-1_0 directly (maiml.xsd / maiml-document.xsd / maiml-protocol.xsd / maiml-data.xsd / maiml-eventLog.xsd / maiml-pnml.xsd / maiml-core.xsd / maiml-property.xsd) -- see the docstring of each write* function for the specific xs:sequence it mirrors.

loads()/load() are the inverse: parse MaiML XML (via lxml, so attribute namespace maps are easy to inspect) back into a maiml_domain object tree, returned wrapped in a LoadedMaiml along with the root's own custom namespace declarations and every id encountered in the file -- both are needed by the "load an existing protocol, keep it, add new data/eventLog" workflow: the namespaces so dumps() can be given the same extra_namespaces= again, and the ids so pymaiml.builders.IdFactory.from_existing_ids() can avoid generating a new id that collides with one already in the file.

Known limitations (documented rather than silently guessed at): - elements reuse the exact same concrete property/content classes as top-level / -- the schema's uncertaintyBaseType is the common abstract ancestor of both propertyBaseType and contentBaseType (see maiml-property.xsd), so a FloatType/ContentFloatListType/etc. instance can be written under either tag. _write_property_or_content()/_read_property_or_content() take an optional tag= override for exactly this purpose. - EncryptionType's encrypted_data is stored (by maiml_domain) as a raw XML string on both the way in and the way out; it is not decrypted, inspected, or re-encrypted. - Custom key prefixes (e.g. "KYL:Cantilever", "ISO18115-3:Wavenumber") must have their namespace declared via extra_namespaces when writing -- MaiML's key attributes are xs:QName, which XSD validation rejects if the prefix has no xmlns declaration in scope. loads() reports whatever was declared on the root element via LoadedMaiml.namespaces so a load-modify-dump round trip can reuse it without the caller having to re-track it by hand. - document.signature (a read back by loads()) is read for inspection but never re-emitted by dumps()/dump() -- see dumps()'s docstring for why. If you need a signed output, sign the bytes dumps() produces with a dedicated external tool, after dumping, not before. - loads() picks out only the specific XSD elements/attributes it knows about; it does not model XML comments or processing instructions at all. A load()/dump() round trip therefore silently drops any comments or processing instructions the original file had -- this is not a lossless-XML-round-trip guarantee, and a comment can carry meaningful information (e.g. "this field was intentionally left blank"), not just a developer note, so this loss is worth calling out explicitly rather than treating it as an obvious side effect of pretty-printing.

LoadedMaiml dataclass

LoadedMaiml(root: Union['m.MaimlRootType', 'm.ProtocolFileRootType'], namespaces: Dict[str, str] = dict(), ids: List[str] = list())

Result of loads()/load(): the parsed object tree plus the two things a "load an existing file, keep its protocol, add new data/eventLog" workflow needs and can't get from the maiml_domain objects alone.

root: a MaimlRootType or ProtocolFileRootType, matching the file's own xsi:type. For a ProtocolFileRootType, build a new MaimlRootType re-using root.document and root.protocol as-is, plus your own new DataType/EventLogType, then dumps() that. namespaces: every xmlns: declared on the root element, other than the fixed default (MAIML_NS) and xsi: bindings -- e.g. {"KYL": "...", "lifecycle": "http://www.xes-standard.org/..."}. Pass this straight back as dumps(..., extra_namespaces=namespaces) so a load-modify-dump round trip doesn't have to re-track which custom prefixes the original file declared. ids: every id= value found anywhere in the file, in document order. Feed this to pymaiml.builders.IdFactory.from_existing_ids(ids) so newly generated ids for the data you add cannot collide with ids already used by the loaded content.

dumps

dumps(root_obj: Union['m.MaimlRootType', 'm.ProtocolFileRootType'], *, extra_namespaces: Optional[Dict[str, str]] = None, pretty: bool = True) -> str

Serialize a MaimlRootType/ProtocolFileRootType object tree to a MaiML XML string.

extra_namespaces: {prefix: uri} declared as xmlns: on the root element. Required for any custom key= prefix used on a property/content (e.g. {"KYL": "http://example.org/kyl-instrument-properties"}) -- MaiML's key attributes are xs:QName, so XSD validation fails if the prefix has no namespace declaration in scope. The XES lifecycle/concept/ time extensions, if used via property keys like "lifecycle:transition", must likewise be declared here with their exact standard URIs (http://www.xes-standard.org/.xesext#).

root_obj.document.signature (a loads() read back from an existing file) is ALWAYS dropped from the output, unconditionally -- there is no parameter to keep it. Earlier versions had a drop_stale_signature= parameter that kept a signature through when dumps() could tell nothing besides the signature had changed since load; that has been removed.

The reason is not merely "an edited file's signature is stale" -- it's that dumps() cannot make ANY serialization of this object tree a safe carrier of a pre-existing enveloped signature, changed or not. MaiML's is an enveloped XML signature under JIS X 5093 / ETSI TS 101 903 (XAdES): the digest is computed over the exact serialized byte form of the document at the moment of signing, and JIS's own signing procedure treats that byte form as fixed afterwards (nothing may be added after the closing tag but a trailing newline). dumps() reconstructs the tree from maiml_domain objects and re-applies its own formatting (indentation, namespace-declaration placement, attribute ordering, empty-element representation, ...); it does not reproduce another implementation's exact byte form, and pymaiml does not implement XAdES signing or verification itself (see CONTRIBUTING.md) -- so it has no way to certify that any particular dumps() output is still a valid carrier for a signature it did not just compute itself. Treating "detectably unchanged content" as grounds for keeping the old signature (the previous behavior) implied a safety guarantee pymaiml cannot actually make.

Practically: pymaiml.serialization.loads() still reads root_obj.document.signature back for inspection (e.g. to hand to an external verifier), but dumps()/dump() never write it back out. If you need a signed MaiML file, dump the content first, then sign the resulting bytes with a dedicated external tool -- treat "build/edit the MaiML content" and "sign the finished file" as two separate steps, in that order, never the other way around.

Corollary (see CONTRIBUTING.md's "XML Signature" section): once a MaiML file IS signed, never run its bytes through pretty-printing, comment stripping, whitespace collapsing, line-ending conversion (CRLF/LF -- watch for tools/git settings that silently do this), or any other reformatting when merely saving/copying it -- any of those changes the canonicalization result the signature was computed over, even though pymaiml itself never re-emits a Signature it did not just compute.

ソースコード位置： pymaiml/serialization.py

def dumps(
    root_obj: Union["m.MaimlRootType", "m.ProtocolFileRootType"],
    *,
    extra_namespaces: Optional[Dict[str, str]] = None,
    pretty: bool = True,
) -> str:
    """
    Serialize a MaimlRootType/ProtocolFileRootType object tree to a MaiML
    XML string.

    extra_namespaces: {prefix: uri} declared as xmlns:<prefix> on the root
    <maiml> element. Required for any custom key= prefix used on a
    property/content (e.g. {"KYL": "http://example.org/kyl-instrument-properties"})
    -- MaiML's key attributes are xs:QName, so XSD validation fails if the
    prefix has no namespace declaration in scope. The XES lifecycle/concept/
    time extensions, if used via property keys like "lifecycle:transition",
    must likewise be declared here with their exact standard URIs
    (http://www.xes-standard.org/<name>.xesext#).

    root_obj.document.signature (a <Signature> loads() read back from an
    existing file) is ALWAYS dropped from the output, unconditionally --
    there is no parameter to keep it. Earlier versions had a
    drop_stale_signature= parameter that kept a signature through when
    dumps() could tell nothing besides the signature had changed since
    load; that has been removed.

    The reason is not merely "an edited file's signature is stale" -- it's
    that dumps() cannot make ANY serialization of this object tree a safe
    carrier of a pre-existing enveloped signature, changed or not. MaiML's
    <Signature> is an enveloped XML signature under JIS X 5093 / ETSI TS
    101 903 (XAdES): the digest is computed over the exact serialized byte
    form of the document at the moment of signing, and JIS's own signing
    procedure treats that byte form as fixed afterwards (nothing may be
    added after the closing </Signature> tag but a trailing newline).
    dumps() reconstructs the tree from maiml_domain objects and re-applies
    its own formatting (indentation, namespace-declaration placement,
    attribute ordering, empty-element representation, ...); it does not
    reproduce another implementation's exact byte form, and pymaiml does
    not implement XAdES signing or verification itself (see
    CONTRIBUTING.md) -- so it has no way to certify that any particular
    dumps() output is still a valid carrier for a signature it did not
    just compute itself. Treating "detectably unchanged content" as
    grounds for keeping the old signature (the previous behavior) implied
    a safety guarantee pymaiml cannot actually make.

    Practically: pymaiml.serialization.loads() still reads
    root_obj.document.signature back for inspection (e.g. to hand to an
    external verifier), but dumps()/dump() never write it back out. If you
    need a signed MaiML file, dump the content first, then sign the
    resulting bytes with a dedicated external tool --
    treat "build/edit the MaiML content" and "sign the finished file" as
    two separate steps, in that order, never the other way around.

    Corollary (see CONTRIBUTING.md's "XML Signature" section): once a
    MaiML file IS signed, never run its bytes through pretty-printing,
    comment stripping, whitespace collapsing, line-ending conversion
    (CRLF/LF -- watch for tools/git settings that silently do this), or
    any other reformatting when merely saving/copying it -- any of those
    changes the canonicalization result the signature was computed over,
    even though pymaiml itself never re-emits a Signature it did not just
    compute.
    """
    maiml_el = _build_maiml_element(root_obj, extra_namespaces=extra_namespaces)

    rough = ET.tostring(maiml_el, encoding="unicode")
    rough = _dedupe_root_namespace_decls(rough)
    if not pretty:
        return '<?xml version="1.0" encoding="UTF-8"?>\n' + rough

    pretty_xml = minidom.parseString(rough).toprettyxml(indent="  ")
    pretty_xml = "\n".join(line for line in pretty_xml.split("\n") if line.strip())
    pretty_xml = pretty_xml.split("\n", 1)[1]  # drop minidom's own XML declaration
    return '<?xml version="1.0" encoding="UTF-8"?>\n' + pretty_xml

dump

dump(root_obj: Union['m.MaimlRootType', 'm.ProtocolFileRootType'], path: Union[str, Path], **kwargs) -> None

Serialize root_obj and write it to path (parent directories created as needed).

ソースコード位置： pymaiml/serialization.py

def dump(
    root_obj: Union["m.MaimlRootType", "m.ProtocolFileRootType"],
    path: Union[str, Path],
    **kwargs,
) -> None:
    """Serialize root_obj and write it to `path` (parent directories created as needed)."""
    path = Path(path)
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(dumps(root_obj, **kwargs), encoding="utf-8")

loads

loads(xml_text: Union[str, bytes]) -> LoadedMaiml

Parse MaiML XML text (or bytes) into a LoadedMaiml.

xml_text is treated as untrusted input -- it may be a local file's contents today, but this is also the entry point a future API/upload surface would call directly, without necessarily running it through pymaiml.validation.validate() first. Parsing therefore explicitly disables external entity resolution and network access (see pymaiml._xml_security.make_untrusted_input_parser()) rather than relying on lxml's current defaults.

The returned root's document.signature (when the input carried a ) is a re-serialization of that subtree, not a byte-for- byte slice of the input -- see _read_document()'s docstring for why this matters if you intend to verify the signature against it.

ソースコード位置： pymaiml/serialization.py

def loads(xml_text: Union[str, bytes]) -> LoadedMaiml:
    """
    Parse MaiML XML text (or bytes) into a LoadedMaiml.

    xml_text is treated as untrusted input -- it may be a local file's
    contents today, but this is also the entry point a future API/upload
    surface would call directly, without necessarily running it through
    pymaiml.validation.validate() first. Parsing therefore explicitly
    disables external entity resolution and network access (see
    pymaiml._xml_security.make_untrusted_input_parser()) rather than
    relying on lxml's current defaults.

    The returned root's document.signature (when the input carried a
    <Signature>) is a re-serialization of that subtree, not a byte-for-
    byte slice of the input -- see _read_document()'s docstring for why
    this matters if you intend to verify the signature against it.
    """
    data = xml_text.encode("utf-8") if isinstance(xml_text, str) else xml_text
    root_el = _lxml_etree.fromstring(data, parser=make_untrusted_input_parser())

    xsi_type = root_el.get(f"{{{XSI_NS}}}type")
    namespaces = {
        prefix: uri
        for prefix, uri in root_el.nsmap.items()
        if prefix is not None and prefix != "xsi"
    }
    all_ids = [el.get("id") for el in root_el.iter() if el.get("id") is not None]

    document_el = _find(root_el, "document")
    document = _read_document(document_el)
    protocol_el = _find(root_el, "protocol")
    protocol = _read_protocol(protocol_el) if protocol_el is not None else None
    features = root_el.get("features")

    if xsi_type == "maimlRootType":
        data_el = _find(root_el, "data")
        event_log_el = _find(root_el, "eventLog")
        root_obj = m.MaimlRootType(
            document=document, protocol=protocol,
            data=_read_data(data_el), event_log=_read_event_log(event_log_el),
            features=features,
        )
    elif xsi_type == "protocolFileRootType":
        root_obj = m.ProtocolFileRootType(document=document, protocol=protocol, features=features)
    else:
        raise ValueError(
            f"Unknown maiml/@xsi:type: {xsi_type!r} (expected 'maimlRootType' or 'protocolFileRootType')"
        )

    return LoadedMaiml(root=root_obj, namespaces=namespaces, ids=all_ids)

load

load(path: Union[str, Path]) -> LoadedMaiml

Read path and parse it via loads().

ソースコード位置： pymaiml/serialization.py

def load(path: Union[str, Path]) -> LoadedMaiml:
    """Read `path` and parse it via loads()."""
    return loads(Path(path).read_bytes())