Skip to content

[Proposal] Adding an optional serialization dependency. #519

Open
@limarta

Description

@limarta

GenSerialization.jl serves to serialize the common trace types offered by Gen.

Why?
Direct serialization using Julia's native serialize function does not properly save ephemeral (e.g. generative function pointers) as pointed out in #129. Users who intend to save out traces for long term storage will necessarily restart the Julia runtime, so there is no guarantee that runtime data will properly be read back in. Given a saved out trace, there are two ways to read data back in. The first requires the generative function to reconstruct the function pointers, and #369 takes a similar strategy and implicitly invokes the generative function while reading. The second amortizes the generative function call over calls to the GFI such as update and regenerate. In the package, we distinguish the two by realize and deserialize respectively.

Goals:

  • Provide out of the box serialization for Dynamic, Static, and combinator traces.
  • Support programs with untraced randomness.

Caveats:

  • Currently supports Dynamic and Map traces. The remaining combinators aren't yet supported, but they are essentially copies of Map.
  • The format, which is opaque to the user, assumes the machine used to read a trace has the same endianness as the machine used to write the trace.
  • This is a Julia-only solution, so it does not provide support to serialize between different Gen implementations.

WIP

  • Finish support of combinators
  • Finish Static DSL traces.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions