# Schema Validation

You may have ended up here because someone has mentioned "The Schema Dance." This is a term we have colloquially adopted
to describe the process of introducing a breaking schema change. But to understand that you should first be aware of
what schema validation is, and why it's helpful.

Let's say we have a GraphQL type in our schema called `Song`, which looks something like:

```graphql
type Song {
  isrc: ID!
  name: String!
}
```

Now suppose you have a new feature request to add cover art to a song, because it's fun to look at cover art. So you
make a change to the `Song` schema type to add a `coverArtImageURL` field:

```graphql
type Song {
  isrc: ID!
  name: String!
  coverArtImageURL: String!
}
```

Your pull request to the GraphQL service is merged, and you start working on adding cover art to your frontend service,
making a GraphQL query

```graphql
query {
  song(isrc: "QM4TW2003053") {
    isrc
    name
    coverArtImageURL
  }
}
```

adding to your `<Song />` React component:

```jsx
const Song = ({ isrc, name, coverArtImageURL }) => (
  <div class="song">
    <img
        class="cover-art"
        src={coverArtImageURL}
    />
    <span class="name">{name}</span>
  </div>
);
```

and begin using this throughout the frontend. A few weeks later, however, someone makes a complaint that... well, we've
been calling this sort of thing `imageLocation` throughout the rest of the codebase, so it would really be more useful
if we could name the field `imageLocation` on the `Song` type. Seems a little nit-picky if you ask me, but fine. So
you change the GraphQL schema (as well as your resolver definition):

```graphql
type Song {
  isrc: ID!
  name: String!
  imageLocation: String!
}
```

and submit a pull request. The only problem is, the pipeline is broken. For some reason a step called
"Push QA Schema" is failing, and you see this in the logs:

```
→ breaking changes found
╔════════╤═══════════════╤══════════════════════════════════════════════════════════╗
║ Change │ Code          │ Description                                              ║
╟────────┼───────────────┼──────────────────────────────────────────────────────────╢
║ FAIL   │ FIELD_REMOVED │ type `Song`: field `coverArtImageURL` removed            ║
╟────────┼───────────────┼──────────────────────────────────────────────────────────╢
║ PASS   │ FIELD_ADDED   │ type `Song`: field `imageLocation` added                 ║
╚════════╧═══════════════╧══════════════════════════════════════════════════════════╝
```

What went wrong?!

## Apollo Studio Schema Validation

Any time a pull request is merged to one of our GraphQL microservices, there's a step in the pipeline that validates the
schema to make sure that it's not going to cause any of our existing clients to experience issues. In the case above,
we removed the `coverArtImageURL` even though our client's React component was still depending on it. It's nice that
this got caught, but how?

After every merge and deployment of a GraphQL service, there's a step in the pipeline which will update the prod
schema. This takes a snapshot of all the types, fields, field variables, etc. related to the GraphQL schema, and sends
it to Apollo Studio. Before any schema changes can be applied, the pipeline will check with Apollo Studio to validate
that nothing is being removed from the schema that could cause failures (these are only _theoretical_ failures, Apollo
Studio doesn't know if the fields are currently being read by clients, just that they've been used by a client in the
last week).

### Resolving Validation Errors

The proper way to migrate from the previous field to the new one is do the following:

#### 1. Add the new field name, while marking the old one as deprecated.

This involves a pull request that will create support for both the old and new field names simultaneously:

```graphql
type Song {
  isrc: ID!
  name: String!
  coverArtImageURL: String! @deprecated(reason: "Use imageLocation")
  imageLocation: String!
}
```

#### 2. Adjust clients to accept the new field name

This would involve making modifications to our frontend client to reflect the new name in the GraphQL query, as well as
in our React component:

```graphql
query {
  song(isrc: "QM4TW2003053") {
    isrc
    name
    imageLocation
  }
}
```

```jsx
const Song = ({ isrc, name, imageLocation }) => (
  <div class="song">
    <img
        class="cover-art"
        src={imageLocation}
    />
    <span class="name">{name}</span>
  </div>
);
```

#### 3. Remove the old (and deprecated) field

A pull request to remove the old field can be opened immediately, but it will fail the Apollo Studio Validation checks.
In most cases, it is possible just to wait for the field to not have been used in the past week. At this point, you can
request the jenkins checks to be rerun and they should pass and allow the removal of the deprecated field.

## The Schema Dance 🕺💃

In some circumstances, it may be necessary to ignore the validation checks for some PRs. An example is if you have
performed the above steps, but need to remove the deprecated field sooner than the week duration imposed by the
validation checks. In this case you can manually override the checks. If you are not sure whether you should dance a
schema change in, feel free to reach out to the GraphQL Maintainers - they can advise you.

### Let the Dance Begin!

Given the above example, let's assume we've converted our client (and checked to see that there are no other clients
using the `@deprecated` field we're getting rid of), and we know that it's "safe" to get rid of the field we were using
before. In order to push these changes into production, we'll need to do the schema dance! This involves a careful
coordination of doing the following:

1. Announce to [#graphql](https://orcd.slack.com/archives/CE2MVK8BF) that you're "about to do the schema dance"
2. Disable the schema validation steps in your GraphQL service pipeline so that the schema is not validated
3. Run your GraphQL service pipeline to ensure that the "breaking" changes to the schema have been applied
4. Enable the schema validation steps in your GraphQL service pipeline so that the schema is validated once again

The critical things to remember here are:

* **Please tell [#graphql](https://orcd.slack.com/archives/CE2MVK8BF) that you are doing the schema dance.** Remember
that while the dance is ongoing there are no checks that code being deployed will validate that the schema is valid. A
careless pull request that is merged during this time could do all sorts of things that would break clients, so it's
important to let everyone know that you're about to perform this delicate... dance.

* **Ensure that you enable the validation steps after you are done.** If you don't do this, there won't be any schema
validation in the pipeline whatsoever, which is worse than forgetting to announce that you're doing the schema dance,
because now the schema safeguard is off until someone remembers to turn it back on!

## Federation Validation Failures

In addition to the above simple validation for fields/types there are a set of validation rules that prevent you from
doing things that would result in an invalid composed graph. An example of this is where you are wanting to move a
field between two services. There are other more complex situations where you may need to update more than two services
simultaneously, for example if you update a [Shared Value Type](../../principles/federation#Value%20Types), you must update every service that uses that type in its
schema.

Say we have service A with this type:

```graphql
type SharedType @key(fields: "id") {
  id: ID!
  name: String
}
```

And service B with this type:

```graphql
extend type SharedType @key(fields: "id") {
  id: ID! @external
}
```

And we want to migrate the `name` field from service A to service B with zero downtime.

We would raise a PR against service A to add the field:

```diff
 type SharedType @key(fields: "id") {
   id: ID!
-  name: String
 }
```

This would fail validation due to the removal of a field that is in use.

Simultaneously, we can raise a PR against service B to add the field:

```diff
 extend type SharedType @key(fields: "id") {
   id: ID!
+  name: String
 }
```

In this case, we will get a federation validation error because the checks will see that if this PR was merged on its
own, we would end up with two services owning the same field - a condition prohibited by federation to avoid ambiguity.

So, neither of our PRs are independently valid, but together they form a valid schema change. For this, we need to do
a Federated Schema Dance!

## Federated Schema Dance

Much like the regular schema dance, this operation has the opportunity to cause problems for GraphQL clients. This
version will require significant changes to Jenkins pipelines in order to go through successfully, and will block any
concurrent deployments to GraphQL services in the meantime. Ideally for fields that are going to be moved between
services you should have a suitable suite of integration tests at the gateway level.

Firstly, make sure you announce to [#graphql](https://orcd.slack.com/archives/CE2MVK8BF) that you're "about to do the
federated schema dance".

The general rule of thumb is that the Gateway should always be able to resolve fields correctly given its current
representation of the graph. A Composition Failure will **not** result in the representation of the graph being updated.

Therefor, we are able to enter a state of Composition Failure deliberately so that we can then resolve it.

For the example above the basic outline of the steps required is as follows:

1. Merge & deploy the PR to Service B - this puts our composed graph in to an invalid state. The `name` field will still
   be resolved by going to Service A at this point.
2. Merge the PR for Service A, pushing the federated schema but **not** deploying the service change. We need to make
   sure to push the federated schema before we deploy the code change as otherwise there would be a time when the
   gateway would be attempting to resolve the `name` field from Service A while the deployed service was not able to
   resolve that field. After this step, the Gateway should be resolving the `name` field from service B
3. The service change for service A can then be deployed normally, as it would no longer be being called for the `name`
   field.

These steps should be done once for QA and then repeated for Prod after confirming that everything is OK.

A more in depth set of instructions can be found below:

### Merging Service B change

In this case, we should adjust the service B pipeline to comment out the QA Schema Validation step, and all steps after
integration tests.

The PR can then be merged.

The pipeline will fail at the 'Deploy Schema' step after merging the PR.

### Merging Service A change

Before merging the PR, we should comment out the QA Schema Validation and the service deployment steps, as well as all
steps after the integration tests.

The PR can then be merged.

The pipeline should succeed.

You can then un-comment the QA Service Deployment step and re-run the pipeline, which should also run successfully,
including all integration tests.

You should now check that the QA graph is healthy and responds as you expect.

### Running Service B change to production

You can then run the service B change to production - all QA steps should be un-commented, only the production
Schema Validation step should be commented here.

Running this pipeline should fail at deploying the production schema.

### Running Service A change to production

The Service A pipeline should have all QA steps un-commented, and the production Schema Validation and service
deployment steps should be commented out.

Running this pipeline should succeed, causing the production Gateway to re-point the `name` field resolution to service
B.

The Service A pipeline should then be ran with all steps un-commented. This should succeed.

### Cleanup Service B pipeline

Service B should then have its pipeline Schema Validation steps un-commented, and the pipeline should be reran. This
should succeed.

The Federated Schema Dance is then complete.