Writing GatsbyJS configuration in Typescript

SoftwarePhilosophy
2019-11-21

By moving GatsbyJS configuration to a typed language the implementation can be checked by the computer

tldr;

Require ts-node to allow requiring ts files worked example by Dave Clark

Problem Context

When configuring gatsby several files exist outside the standard plugin ecosystem, most urgently gatsby-node.js which is used to define how content maps to nodes maps to pages. This site is mostly written in markdown in a content/ directory. Through several plugins the markdown is transformed into a visual representation, roughly:

  1. gatsby-source-filesystem loads files
  2. gatsby-transformer-remark takes *.md files and transforms them into abstract nodes with several properties like html, excerpt, frontmatter
  3. gatsby-node.js queries the nodes and builds navigation structure based on frontmatter (title, parent, date, path)
  4. It then runs createPage on some nodes assigning them a url, template and passing in computed data via pageContext
  5. When the users browser visits the url the template loads the node and uses react to render markup

The transformations run by gatsby-node are highly dependent on the exact structure of graphql results. It can be difficult to mentally verify that several layers of recursive logic on complex objects has been done correctly and often results in fragile code that was tested until it works then walked away from.

Potential solution

The methodology of type driven development roughly boils down to stating the problem using types, then writing implementation until the type-checker passes.

For example, stating the problem of adding next and previous buttons to navigate between siblings of articles based on parentPath can be done as follows:

interface Article {
  title: string;
  path: string;
  parentPath?: string;
}

interface PageContext {
  nextArticle: Article | null;
  prevArticle: Article | null;
}

function buildContext(articles: Article[]): PageContext {
  return {}
  // Returns error "Type '{}' is missing the following properties from type 'PageContext': nextArticle, prevArticle"
}

GatsbyJS fix

Following this github issue and this gist gatsby-node.js was updated to contain:

require("source-map-support").install()
require("ts-node").register()

exports.createPages = require("./src/gatsby/create-pages").createPages

./src/gatsby/create-pages.ts looks similar to:

import { GatsbyNode } from "gatsby"

export const createPages: GatsbyNode["createPages"] = async ({
  actions,
  graphql,
  reporter,
}) => {
    ...
}

After copying the existing createPages code into a typescript file many assumptions about what properties were available on objects were unveiled, meaning that with an unexpected change of input the code would start falling over. The process of annotating the function with types was time consuming and educational, resulting in a more clear understanding of the semantics of the code.

One downside to this process is that the initial assumptions (type of the output of a graphql query) are hand written by the developer. If the source data does not match the developers assumptions then the type checking doesn't provide utility.

Note: I was unable to use a graphql fragment in create-pages.ts, there may be a workaround but it will be dealt with later

Generating graphQL types

GatsbyJS is highly dependent on GraphQL, which is usually written as docstrings inline with the rest of the code. Manually writing types for the returned data types is a chore and prone to human error.

Using gatsby-plugin-typegen the graphql types can be generated, maintaining their accuracy and usefulness.

Fixing issues

The first issue encountered is known by the plugin author

Cannot find module 'graphql-tag-pluck' at /home/username/projects/specific.solutions.limited/site/src/images.d.ts

The fix is straightforward,

yarn add graphql-tag-pluck

The next issue is a stinky one:

AggregateError: GraphQLDocumentError: Unknown fragment "GatsbyImageSharpFixed".

This is also known by the plugin author and I'd like to take a crack at it

Cloning the plugin locally

First I uninstalled the node_modules copy of the plugin and cloned the project into the local projects directory

yarn remove gatsby-plugin-typegen
cd plugins
git clone https://github.com/cometkim/gatsby-plugin-typegen
cd gatsby-plugin-typegen
yarn
yarn build

Attempting to build the gataby site revealed a node module resolution issue:

 ERROR #11321  PLUGIN

"gatsby-plugin-typegen" threw an error while running the onPostBootstrap lifecycle:

Cannot use GraphQLSchema "[object GraphQLSchema]" from another module or realm.

This was fixed manually by creating a soft link

cd plugins/gatsby-plugin-typegen/node-modules
rm -r graphql
cd ../../../
yarn add graphql
ln -s plugins/gatsby-plugin-typegen/node_modules/graphql node_modules/graphql

Likely there is a more idiomatic way to fix this but it is a temporary hack until the fork is pushed to github

Tightening the dev loop

time npm run develop
...
npm run develop  14.34s user 5.91s system 128% cpu 15.758 total

15 seconds to test the code is unacceptable, let's create the simplest possible reproduction

A new gatsby project was created, cleaned up, and setup for typescript, then the above steps were followed to create a local clone of gatsby-plugin-typegen

Reproducing the intended behavior

After some minor battles with the proper order of operations to enable local development the indended behavior of the plugin was reproduced sucessfully.

Minor note: ensure that the generated types are placed OUTSIDE the gatsby ./src directory otherwise they will cause a hot reload loop when they are updated...

import React from "react"
import { graphql } from "gatsby"
import { SiteMetadataPageQuery } from "../../types.generated"
import { useSiteMetadata } from "./hooks"

interface PageData<D> {
  data: D;
}

export default (page: PageData<SiteMetadataPageQuery>) => {
  const metadataStatic = useSiteMetadata()
  return (
    <div>
      {metadataStatic.site.siteMetadata.title} -{" "}
      {page.data.site.siteMetadata.title}
    </div>
  )
}

export const query = graphql`
  query SiteMetadataPage {
    site {
      siteMetadata {
        title
      }
    }
  }
`

Page queries and static queries in the same file

When attempting to put both a page query and a static query in the same file the following error occurred:

 ERROR #85910  GRAPHQL

Multiple "root" queries found in file: "SiteMetadataPage" and "SiteMetadataStatic".

This was worked around by putting useStaticQuery calls in individual files

Reproducing the issue

The redundancy of defining site metadata is obnoxous, so following the fragment tutorial fragments.ts was created

import { graphql } from "gatsby"

export const metadataFragment = graphql`
  fragment SiteMetadataFragment on Site {
    siteMetadata {
      title
    }
  }
`

The naive implementation of fragments did not reproduce the issue, so a more in-depth look at gatsby-transformer-sharp is required

https://github.com/gatsbyjs/gatsby/blob/c175f9ecfb0ed7a4e1d53cc233e3ecdb5801ec3d/packages/gatsby-transformer-sharp/src/fragments.js

Let's create a micro-library to provide this fragment

Figuring out how it copies... https://github.com/gatsbyjs/gatsby/issues/5663

https://github.com/gatsbyjs/gatsby/blob/2.13.30/packages/gatsby/src/query/query-compiler.js

.cache/fragments seems to be important?

While it is proving difficult to get gatsby to accept this fragment perhaps the plugin can be tweaked...

Attempting to duplicate the logic in query-compiler was stanky, lots of code was copy pasted and ts-ignored but overall a success!

Unfortunately the generated types file has duplicate interfaces. This was resolved by omitting the .cache directory from the files list

Pull request created!

Specific Solutions LLC
Portland, OR