Jupyter Book V3 AST: Better Output Handling and Future Possibilities

We’re excited to announce the release of MyST Markdown V3 AST support in Jupyter Book, a significant upgrade that improves how Jupyter notebook outputs are handled and allows for better control over how Jupyter generated content is rendered.

What Changed?¶

In previous versions, all outputs from a Jupyter notebook cell were bundled together as a single data structure. While this worked for basic use cases, it limited our ability to handle individual outputs with the flexibility and precision that modern documentation needs.

With the V3 AST upgrade, each output from a code cell is now represented as its own node in the Abstract Syntax Tree. See the old (left) vs. the new (right) AST structure below:

{
  "type": "output",
  "id": "cell-output-1",
  "data": [
    // Array of all output data from the cell
    { "output_type": "display_data", "data": {...}, ... },
    { "output_type": "stream", "text": "...", ... },
    // ... more outputs
  ]
}

{
  "type": "outputs",
  "id": "cell-outputs-1",
  "children": [
    {
      "type": "output",
      "id": "output-1",
      "jupyter_data": { "output_type": "display_data", "data": {...} },
      "children": []
    },
    {
      "type": "output",
      "id": "output-2",
      "jupyter_data": { "output_type": "stream", "text": "..." },
      "children": []
    }
  ]
}

Why This Matters¶

This change is foundational for several reasons. It brings Jupyter outputs into the MyST AST as first-class citizens, rather than treating them as opaque data blobs. This enables outputs to participate in the same document processing pipeline as the rest of your content.

The new structure allows us to parse and process output content (like Markdown or LaTeX generated by code cells) directly into MyST AST nodes. This means outputs will be able to act like other MyST content: they’ll be able to define and consume reference labels, appear in cross-reference resolution, and integrate seamlessly with the rest of your documentation.

What’s Next?¶

This lays a structural foundation in the AST for us to build in more native MyST functionality into Jupyter outputs. For example, the PR below has follow-up work to enable Markdown or Latex outputs from cells to be parsed into the MyST AST (enabling features like cross-referencing from outputs describe above).

jupyter-book/mystmd#1961

This enhancement proposal contains ongoing discussion among the team about the structure and behavior of MyST AST in code outputs:

jupyter-book/myst-enhancement-proposals#32

Migrating AST versions¶

For users: If you are a user of jupyter-book or mystmd CLIs, you should not have to worry about the upgrade, your existing content will work^[1] after a clean local build. Just run commands like the following:

$ jupyter book clean --all
$ jupyter book start

For developers: If you have developed your own custom “theme” for your books you have two options:

1. Re-build and deploy your content so that it is built with the new AST

2. Integrate the upgrade and downgrade functions from the myst-migrate package into your front-end stack. We recommend the latter, but especially as it will be needed to make full use of MyST Markdown’s remote cross referencing features. See myst-theme for how myst-migrate is used there.

Learn More & Contribute¶

• For technical details about the AST changes and to comment or contribute, see: jupyter-book/myst-enhancement-proposals#32.

• To see the new behavior, upgrade mystmd to mystmd@1.7.0+.