Adding Jupyter Notebook (and Rust) to Gatsby

An insane plan to allow posts on this site to perform computations and display output

What? How?

Jupyter Notebook is an interactive environment where code can be typed into cells and the outputs visualized. evcxr_jupyter is a custom jupyter kernel that can run rust. gatsby-transformer-ipynb is a gatsby plugin that can render jupyter notebooks during site generation. Let's get frankensteining!

Installing Jupyter

This computer is Windows and primary development is in WSL1 (Ubuntu-ish)

sudo apt install python3 python3-pip
pip install --user jupyterlab
which jupyter
> /home/me/.local/bin/jupyter
jupyter lab

stock jupyter — Stock jupyter lab running in a browser

Installing in gatsby

yarn add @gatsby-contrib/gatsby-transformer-ipynb

// In your gatsby-config.js
plugins: [
  {
    resolve: `gatsby-source-filesystem`,
    options: {
      name: `content`,
      path: path.join(__dirname, "content"),
      ignore: [`**/.ipynb_checkpoints`],
    },
  },
  ...`@gatsby-contrib/gatsby-transformer-ipynb`,
]

cd content/blog/03-rusty-jupyter
jupyter lab

example python 3 jupyter post — Testing that python is running and markdown can be rendered

notebook metadata — Adding metadata to the notebook

Correcting upstream bug

yarn develop
...
 ERROR #11321  PLUGIN

"@gatsby-contrib/gatsby-transformer-ipynb" threw an error while running the onCreateNode lifecycle:

Renderer for type `element` not defined or is not renderable

as noted on github 7 days ago (as of writing)

ooorah, let's do this

Using yarn selective dependency resolution the notebook-render dependency is overridden to mine until the PR is accepted:

/// package.json
  "resolutions": {
    "@gatsby-contrib/gatsby-transformer-ipynb/@nteract/notebook-render": "https://github.com/sdobz/notebook-render"
  },

Adding pages

First we need to figure out how the notebook is added to graphql:

graphql explorer — allJupyterNotebook query example

One small distinction is that the front matter of the blog post is titled "metadata"

gatsby-node.js was customized to parse and validate the metadata, details are too complex for the scope of this post.

Creating a template

This is a rough representation of the template used:

const JupyterTemplate = ({
  data: {
    jupyterNotebook: {
      metadata: { title },
      html,
    },
  },
}) => (
  <PageContainer>
    <PageTitle title={title} />
    <Stack dangerouslySetInnerHTML={{ __html: html }} />
  </PageContainer>
)

export default JupyterTemplate

export const query = graphql`
  query JupyterPage($path: String!) {
    jupyterNotebook(metadata: { path: { eq: $path } }) {
      metadata {
        title
      }
      html
    }
  }
`

Viola, a jupyter notebook

Using Rust

cargo install evcxr_jupyter
evcxr_jupyter --install
jupyter lab

Creating a rust notebook...

jupyter notebook with rust kernel — Using a rust kernel in jupyter notebook

all done

Next steps

Currently all previews on this blog have to be markdown, so notebooks will not appear there.

The formatting on the notebooks leaves a bit to be desired and must be improved

Theming between code snippets in markdown and in the notebooks could be unified

Sub posts: