Bindings for mobx-state-tree and GraphQL

Discuss this project on spectrum

Installation: yarn add mobx mobx-state-tree mobx-react react react-dom mst-gql graphql-request

If you want to use graphql tags, also install: yarn add graphql graphql-tag

Watch the introduction talk @ react-europe 2019: Data models all the way by Michel Weststrate

Both GraphQL and mobx-state-tree are model-first driven approaches, so they have a naturally matching architecture. If you are tired of having your data shapes defined in GraphQL, MobX-state-tree and possible TypeScript as well, this project might be a great help!

Furthermore, this project closes the gap between GraphQL and mobx-state-tree as state management solutions. GraphQL is very transport oriented, while MST is great for client side state management. GraphQL clients like apollo do support some form of client-side state, but that is still quite cumbersome compared to the full model driven power unlocked by MST, where local actions, reactive views, and MobX optimized rendering model be used.

Benefits:

• Model oriented
• Type reuse between GraphQL and MobX-state-tree
• Generates types, queries, mutations and subscription code
• Strongly typed queries, mutations, result selectors, and hooks! Auto complete all the things!
• Local views, actions, state and model life-cycles
• Automatic instance reuse
• Built-in support for local storage, caching, query caching, subscriptions (over websockets), optimistic updates
• Idiomatic store organization
• Incremental scaffolding that preserves changes

The mst-gql libraries consists of two parts:

1. Scaffolding
2. A runtime library

The scaffolder is a compile-time utility that generates a MST store and models based on the type information provided by your endpoint. This utility doesn't just generate models for all your types, but also query, mutation and subscription code base on the data statically available.

The runtime library is configured by the scaffolder, and provides entry points to use the generated or hand-written queries, React components, and additional utilities you want to mixin to your stores.

To get started, after installing mst-gql and its dependencies, the first task is to scaffold your store and runtime models based on your graphql endpoint.

To scaffold TypeScript models based on a locally running graphQL endpoint on port 4000, run: yarn mst-gql --format ts http://localhost:4000/graphql. There are several additional args that can be passed to the CLI or put in a config file. Both are detailed below.

Tip: Note that API descriptions found in the graphQL endpoint will generally end up in the generated code, so make sure to write them!

After running the scaffolder, a bunch of files will be generated in the src/models/ directory of your project (or whatever path your provided):

(Files marked ✏ can and should be edited. They won't be overwritten when you scaffold unless you use the force option.)

• index - A barrel file that exposes all interesting things generated
• RootStore.base - A mobx-state-tree store that acts as a graphql client. Provides the following:
  • Storage for all "root" types (see below)
  • The .query, .mutate and .subscribe low-level api's to run graphql queries
  • Generated .queryXXX ,.mutateXXX and .subscribeXXX actions based on the query definitions found in your graphQL endpoint
• ✏ RootStore - Extends RootStore.base with any custom logic. This is the version we actually export and use.
• ✏ ModelBase - Extends mst-gql's abstract model type with any custom logic, to be inherited by every concrete model type.
• XXXModel.base mobx-state-tree types per type found in the graphQL endpoint. These inherit from ModelBase and expose the following things:
  • All fields will have been translated into MST equivalents
  • A xxxPrimitives query fragment, that can be used as selector to obtain all the primitive fields of an object type
  • (TypeScript only) a type that describes the runtime type of a model instance. These are useful to type parameters and react component properties
• ✏ XXXModel - Extends XXXModdel.base with any custom logic. Again, this is the version we actually use.
• reactUtils. This is a set of utilities to be used in React, exposing the following:
  • StoreContext: a strongly typed React context, that can be used to make the RootStore available through your app
  • useQuery: A react hook that can be used to render queries, mutations etc. It is bound to the StoreContext automatically.

The following graphQL schema will generate the store and message as shown below:

type User {
  id: ID
  name: String!
  avatar: String!
}
type Message {
  id: ID
  user: User!
  text: String!
}
type Query {
  messages: [Message]
  message(id: ID!): Message
  me: User
}
type Subscription {
  newMessages: Message
}
type Mutation {
  changeName(id: ID!, name: String!): User
}

MessageModel.base.ts (shortened):

export const MessageModelBase = ModelBase.named("Message").props({
  __typename: types.optional(types.literal("Message"), "Message"),
  id: types.identifier,
  user: types.union(types.undefined, MSTGQLRef(types.late(() => User))),
  text: types.union(types.undefined, types.string)
})

RootStore.base.ts (shortened):

export const RootStoreBase = MSTGQLStore.named("RootStore")
  .props({
    messages: types.optional(types.map(types.late(() => Message)), {}),
    users: types.optional(types.map(types.late(() => User)), {})
  })
  .actions((self) => ({
    queryMessages(
      variables?: {},
      resultSelector = messagePrimitives,
      options: QueryOptions = {}
    ) {
      // implementation omitted
    },
    mutateChangeName(
      variables: { id: string; name: string },
      resultSelector = userPrimitives,
      optimisticUpdate?: () => void
    ) {
      // implementation omitted
    }
  }))

(Yes, that is a lot of code. A lot of code that you don't have to write 😇)

Note that the mutations and queries are now strongly typed! The parameters will be type checked, and the return types of the query methods are correct. Nonetheless, you will often write wrapper methods around those generated actions, to, for example, define the fragments of the result set that should be retrieved.

To prepare your app to use the RootStore, it needs to be initialized, which is pretty straight forward, so here is quick example of what an entry file might look like:

// 1
import React from "react"
import * as ReactDOM from "react-dom"
import "./index.css"

import { App } from "./components/App"

// 2
import { createHttpClient } from "mst-gql"
import { RootStore, StoreContext } from "./models"

// 3
const rootStore = RootStore.create(undefined, {
  gqlHttpClient: createHttpClient("http://localhost:4000/graphql")
})

// 4
ReactDOM.render(
  <StoreContext.Provider value={rootStore}>
    <App />
  </StoreContext.Provider>,
  document.getElementById("root")
)

// 5
window.store = rootStore

1. Typical react stuff, pretty unrelated to this library
2. Bunch of imports that are related to this lib :)
3. When starting our client, we initialize a rootStore, which, in typical MST fashion, takes 2 arguments:
   1. The snapshot with the initial state of the client. In this case it is undefined, but one could rehydrate server state here, or pick a snapshot from localStorage, etc.
   2. The transportation of the store. Either gqlHttpClient, gqlWsClient or both need to be provided.
4. We initialize rendering. Note that we use StoreContext.Provider to make the store available to the rest of the rendering three.
5. We expose the store on window. This has no practical use, and should be done only in DEV builds. It is a really convenient way to quickly inspect the store, or even fire actions or queries directly from the console of the browser's developer tools. (See this talk for some cool benefits of that)

Now, we are ready to write our first React components that use the store! Because the store is a normal MST store, like usual, observer based components can be used to render the contents of the store.

However, mst-gql also provides the useQuery hook that can be used to track the state of an ongoing query or mutation. It can be used in many different ways (see the details below), but here is a quick example:

import React from "react"
import { observer } from "mobx-react"

import { Error, Loading, Message } from "./"
import { useQuery } from "../models/reactUtils"

export const Home = observer(() => {
  const { store, error, loading, data } = useQuery((store) =>
    store.queryMessages()
  )
  if (error) return <Error>{error.message}</Error>
  if (loading) return <Loading />
  return (
    <ul>
      {data.messages.map((message) => (
        <Message key={message.id} message={message} />
      ))}
    </ul>
  )
})

Important: useQuery should always be used in combination with observer from the "mobx-react" or "mobx-react-lite" package! Without that, the component will not re-render automatically!

The useQuery hook is imported from the generated reactUtils, and is bound automatically to the right store context. The first parameter, query, accepts many different types of arguments, but the most convenient one is to give it a callback that invokes one of the query (or your own) methods on the store. The Query object returned from that action will be used to automatically update the rendering. It will also be typed correctly when used in this form.

The useQuery hook component returns, among other things, the store, loading and data fields.

If you just need access to the store, the useContext hook can be used: useContext(StoreContext). The StoreContext can be imported from reactUtils as well.

Mutations work very similarly to queries. To render a mutation, the useQuery hook can be used again. Except, this time we start without an initial query parameter. We only set it once a mutation is started. For example the following component uses a custom toggle action that wraps a graphQL mutation:

import * as React from "react"
import { observer } from "mobx-react"

import { useQuery } from "../models/reactUtils"

export const Todo = observer(({ todo }) => {
  const { setQuery, loading, error } = useQuery()
  return (
    <li onClick={() => setQuery(todo.toggle())}>
      <p className={`${todo.complete ? "strikethrough" : ""}`}>{todo.text}</p>
      {error && <span>Failed to update: {error}</span>}
      {loading && <span>(updating)</span>}
    </li>
  )
})

The Todo model used in the above component is defined as follows:

export const TodoModel = TodoModelBase.actions((self) => ({
  toggle() {
    return self.store.mutateToggleTodo({ id: self.id }, undefined, () => {
      self.complete = !self.complete
    })
  }
}))

There are few things to notice:

1. Our toggle action wraps around the generated mutateToggleTodo mutation of the base model, giving us a much more convenient client api.
2. The Query object created by mutateToggleTodo is returned from our action, so that we can pass it (for example) to the setQuery as done in the previous listing.
3. We've set the third argument of the mutation, called optimisticUpdate. This function is executed immediately when the mutation is created, without awaiting it's result. So that the change becomes immediately visible in the UI. However, MST will record the patches. If the mutation fails in the future, any changes made inside this optimisticUpdate callback will automatically be rolled back by reverse applying the recorded patches!

Mutations and queries take as second argument a result selector, which defines which objects we want to receive back from the backend. Our mutateToggleTodo above leaves it to undefined, which defaults to querying all the shallow, primitive fields of the object (including __typename and id).

However, in the case of toggling a Todo, this is actually overfetching, as we know the text won't change by the mutation. So instead we can provide a selector to indicate that we we are only interested in the complete property: "__typename id complete". Note that we have to include __typename and id so that mst-gql knows to which object the result should be applied!

Children can be retrieved as well by specifying them explicitly in the result selector, for example: "__typename id complete assignee { __typename id name }. Note that for children __typename and id (if applicable) should be selected as well!

It is possible to use gql from the graphql-tag package. This enables highlighting in some IDEs, and potentially enables static analysis.

However, the recommended way to write the result selectors is to use the query builder that mst-gql will generate for you. This querybuilder is entirely strongly typed, provides auto completion and automatically takes care of __typename and id fields. It can be used by passing a function as second argument to a mutation or query. That callback will be invoked with a querybuilder for the type of object that is returned. With the querybuilder, we could write the above mutation as:

export const TodoModel = TodoModelBase.actions((self) => ({
  toggle() {
    return self.store.mutateToggleTodo({ id: self.id }, (todo) => todo.complete)
  }
}))

To select multiple fields, simply keep "dotting", as the query is a fluent interface. For example: user => user.firstname.lastname.avatar selects 3 fields.

Complex children can be selected by calling the field as function, and provide a callback to that field function (which in turn is again a query builder for the appropriate type). So the following example selector selects the timestamp and text of a message. The name and avatar inside the user property, and finally also the likes properties. For the likes no further subselector was specified, which means that only __typename and id will be retrieved.

// prettier-ignore
msg => msg
  .timestamp
  .text
  .user(user => user.name.avatar)
  .likes()
  .toString()

To create reusable query fragments, instead the following syntax can be used:

import { selectFromMessage } from "./MessageModel.base"

// prettier-ignore
export const MESSAGE_FRAGMENT = selectFromMessage()
  .timestamp
  .text
  .user(user => user.name.avatar)
  .likes()
  .toString()

You can customize all of the defined mst types: RootStore, ModelBase, and every XXXModel.

However, some files (including but not limited to .base files) should not be touched, as they probably need to be scaffolded again in the future.

Thanks to how MST models compose, this means that you can introduce as many additional views, actions and props as you want to your models, by chaining more calls unto the model definitions. Those actions will often wrap around the generated methods, setting some predefined parameters, or composing the queries into bigger operations.

Example of a generated model, that introduces a toggle action that wraps around one of the generated mutations:

// src/models/TodoModel.js
import { TodoModelBase } from "./TodoModel.base"

export const TodoModel = TodoModelBase.actions((self) => ({
  toggle() {
    return self.store.mutateToggleTodo({ id: self.id })
  }
}))

That's it for the introduction! For the many different ways in which the above can applied in practice, check out the examples

There is an exported function called getDataFromTree which you can use to preload all queries, note that you must set ssr: true as an option in order for this to work

async function preload() {
  const client = RootStore.create(undefined, {
    gqlHttpClient: createHttpClient("http://localhost:4000/graphql"),
    ssr: true
  })
  const html = await getDataFromTree(<App client={client} />, client)
  const initalState = getSnapshot(client)

  return [html, initalState]
}

Because you can control what data is fetched for a model in graphql and mst-gql it is possible for a model to have some fields that have not yet been fetched from the server. This can complicate things when we're talking about a field that can also be "empty". To help with this a field in mst-gql will be undefined when it has not been fetched from the server and, following graphql conventions, will be null if the field has been fetched but is in fact empty.

mst-gql generates model types for every object type in your graphql definition. (Except for those excluded using the excludes flag). For any query or mutation that is executed by the store, the returned data will be automatically, and recursively parsed into those generated MST models. This means that for any query, you get a 'rich' object back. Finding the right model type is done based on the GraphQL meta field __typename, so make sure to include it in your graphql queries!

The philosophy behind MST / mst-gql is that every 'business concept' should exist only once in the client state, so that there is only one source of truth for every message, usage, order, product etc. that you are holding in memory. To achieve this, it is recommended that every uniquely identifyable concept in your application, does have an id field of the graphQL ID type. By default, any object types for which this is true, is considered to be a "root type".

Root types have few features:

1. It is guaranteed that any data related to the same id will be updating the very same MST model instance.
2. All instances of root types are stored on the RootStore, for quick and easy lookups.
3. If an object is referring to a root type, a true MST types.reference will be used to establish the reference. This means you can use deep fields in the UI, like message.author.name, despite the fact that this data is stored normalized in the store.
4. Instances of the root types, and all their children, are cached automatically in the root store (until removed manually).

GraphQL has no explicit distinction between compositional and associative relationships between data types. In general, references between graphQL objects are dealt with as follows.

1. If an object is referring to a root type, a types.reference is used, e.g.: author: types.reference(UserModel)
2. If an object is not referring to a root type, but a matching MST model type exist, a composition relationship is used, for example: comments: types.array(CommentModel)
3. If no model type is known for the queried object type, a types.frozen is used, and the data as returned from the query is stored literally.

GraphQL makes it possible to query a subset of the fields of any object. The upside of this is that data traffic can be minimized. The downside is that it cannot be guaranteed that any object is loaded in its 'complete' state. It means that fields might be missing in the client state, even though are defined as being mandatory in the original graphQL object type! To verify which keys are loaded, all models expose the hasLoaded(fieldName:string):boolean view, which keeps track of which fields were received at least once from the back-end.

As described above, (root) model instances are kept alive automatically. Beyond that, mst-gql also provides caching on the network level, based on the query string and variables, following the policies of the apollo and urql graphQL clients. The following fetch policies are supported:

• `"cache-first": Use cache if available, avoid network request if possible
• `"cache-only": Use cache if available, or error if this request was not made before
• `"cache-and-network": Use cache, but still send request and update cache in the background
• `"network-only": Skip cache, but cache the result
• `"no-cache": Skip cache, and don't cache the response either

The default policy is cache-and-network. This is different from other graphQL clients. But since mst-gql leverages the MobX reactivity system, this means that, possibly stale, results are shown on screen immediately if a response is in cache, and that the screen will automatically update as soon as a new server response arrives.

The query cache is actually stored in MST as well, and can be accessed through store.__queryCache.

Since the query cache is stored in the store, this means that mixins like useLocalStore will serialize them. This will help significantly in building offline-first applications.

The mst-gql command currently accepts the following arguments:

• --format ts|js|mjs The type of files that need to be generated (default: js)

• --outDir <dir> The output directory of the generated files (default: src/models)

• --excludes 'type1,type2,typeN' The types that should be omitted during generation, as we are not interested in for this app.

• --roots 'type1,type2,typeN' The types that should be used as (root types)[#root-types]

• --modelsOnly Generates only models, but no queries or graphQL capabilities. This is great for backend usage, or if you want to create your own root store

• --noReact doesn't generate the React related utilities

• --force When set, exiting files will always be overridden. This will drop all customizations of model classes!

• --dontRenameModels By default generates model names from graphql schema types that are idiomatic Javascript/Typescript names, ie. type names will be PascalCased and root collection names camelCased. With --dontRenameModels the original names - as provided by the graphql schema - will be used for generating models.

• --useIdentifierNumber Specifies the use of identifierNumber instead of identifier as the mst type for the generated models IDs. This requires your models to use numbers as their identifiers. See the mobx-state-tree for more information.

• --fieldOverrides id:uuid:idenfitier,*:ID:identifierNumber Overrides default MST types for matching GraphQL names and types. The format is gqlFieldName:gqlFieldType:mstType. Supports full or partial wildcards for fieldNames, and full wildcards for fieldTypes. Case Sensitive. If multiple matches occur, the match with the least amount of wildcards will be used, followd by the order specified in the arg list if there are still multiple matches. Some examples:

  • *_id:*:string - Matches any GQL type with the field name *_id (like user_id), and uses the MST type types.string

  • *:ID:identifierNumber - Matches any GQL type with any field name and the ID type, and uses the MST type types.identifierNumber

  • User.user_id:ID:number - Matches the user_id field on User with the GQL type ID, and uses the MST type types.number

    Specifying this argument additionaly allows the use of multiple IDs on a type. The best matched ID will be used, setting the other IDs to types.frozen()

  • Book.author_id:ID:identifierNumber - Matches the author_id field on Book with the GQL type ID and uses the MST type types.identifierNumber, and sets any other GQL IDs on Book to types.frozen()

    For TS users, input types and query arguments will only be modified for fieldOverrides with a wildcard for gqlFieldName (*:uuid:identifier). An override like *_id:uuid:identifier will not affect input types.

    The primary use case for this feature is for GQL Servers that don't always do what you want. For example, Hasura does not generate GQL ID types for UUID fields, which causes issues when trying to reference associate types in MST. To overcome this, simply specify --fieldOverrides *:UUID:identifier

  • *.timestamp:*:DateScalar:../scalars - Matches any GQL type with the field name timestamp, and uses the MST type DateScalar imported from file ../scalars. Usually used for graphql custom scalar support with MST type.custom

• source The last argument is the location at which to find the graphQL definitions. This can be

  • a graphql endpoint, like http://host/graphql
  • a graphql files, like schema.graphql
  • a parsed graphql file, like schema.json

mst-gql also supports cosmiconfig as an alternative to using cli arguments.

A sample config can be found in Example 2.

The generated RootStore exposes the following members:

Makes a graphQL request to the backend. The result of the query is by default automatically normalized to model instances as described above. This method is also used by all the automatically scaffolded queries.

• The query parameter can be a string, or a graphql-tag based query.
• Variables are the raw JSON data structures that should be send as variable substitutions to the backend. This parameter is optional.
• Options is an optional QueryOptions object. The defaults are fetchPolicy: "cache-and-network" and noSsr: false
• The method returns a Query that can be inspected to keep track of the request progress.

Be sure to at least select __typename and id in the result selector, so that mst-gql can normalize the data.

Similar to query, but used for mutations. If an optimisticUpdate thunk is passed in, that function will be immediately executed so that you can optimistically update the model. However, the patches that are generated by modifying the tree will be stored, so that, if the mutation ultimately fails, the changes can be reverted. See the Optimistic updates section for more details.

Similar to query, but sets up an websocket based subscription. The gqlWsClient needs to be set during the store creation to make this possible. onData can be provided as callback for when new data arrives.

Example initalization:

import { SubscriptionClient } from "subscriptions-transport-ws"

build a websocket client:

// see: https://www.npmjs.com/package/subscriptions-transport-ws#hybrid-websocket-transport
const gqlWsClient = new SubscriptionClient(constants.graphQlWsUri, {
  reconnect: true,
  connectionParams: {
    headers: { authorization: `Bearer ${tokenWithRoles}` }
  }
})

add the ws client when creating the store:

// see: https://github.com/mobxjs/mst-gql/blob/master/src/MSTGQLStore.ts#L42-L43
const store = RootStore.create(undefined, {
  gqlHttpClient,
  gqlWsClient
})

When using server side rendered tools like gatsby/next/nuxt it is necessary to prevent using subscriptions server side. An error will occur because the server is missing a websocket implementation. See code example for gatsby.

Based on the queries, mutations and subscriptions defined at the endpoint, mst-gql automatically scaffolds methods for those onto the base root store.

This is very convenient, as you might not need to write any graphQL queries by hand yourself in your application. Beyond that, the queries now become strongly typed. When using TypeScript, both the variables and the return type of the query will be correct.

An example signature of a generated query method is:

queryPokemons(variables: { first: number }, resultSelector = pokemonModelPrimitives, options: QueryOptions = {}): Query<PokemonModelType[]>

All parameters of this query are typically optional (unless some of the variables are requires, like in the above example).

The result selector defines which fields should fetched from the backend. By default mst-gql will fetch __typename, ID and all primitive fields defined in the model, but full free to override this to make more fine tuned queries! For better reuse, consider doing this in a new action on the appropiate model. For example a query to fetch all comments and likes for a message could look like:

import { MessageBaseModel } from "./MessageModel.base"

const MessageModel = MessageBaseModel.actions((self) => ({
  queryCommentsAndLikes(): Query<MessageModelType> {
    return store.queryMessage(
      { id: self.id },
      `
      id
      __typename
      comments {
        id
        __typename
        text
        likes {
          __typename
          author
        }
      }
    `
    )
  }
}))

• Not a method, but RootStoreType can be used for all places in TypeScript where you need the instance type of the RootStore.
• rawRequest(query: string, variables: any): Promise. Makes a direct, raw, uncached, request to the