GraphQL Configuration Protocol

[asiandrummer]

GraphQL Configuration protocol

In a nutshell, the simplest format is:

export type GraphQLConfiguration = {
  schemaPath?: FilePath,  // may be .json or .graphql
  schemaUrl?: URL,
  schemaJS?: ModulePath | ModuleName,
  // If env is specified, use one of the app configs in here
  env?: GraphQLConfigurationEnvs

  // For multiple applications with overlapping files, bottom configuration options may be helpful
  includeDirs?: Array<DirPath>,
  excludeDirs?: Array<DirPath>,

  // If you have customized validation rules to run
  customValidationRules?: FilePath | Array<FilePath>,

  // If you'd like to specify any other configurations, we provide a reserved namespace for it
  extensions?: GraphQLConfigurationExtension
};

export type GraphQLConfigurationEnvs = {
  production: GraphQLConfiguration,
  development: GraphQLConfiguration,
  // but really just the below
  [envVarName: string]: GraphQLConfiguration
};

export type GraphQLConfigurationExtension = {
  [toolName: string]: any,
};

Note that I'd like to discourage the recursive pattern in using the EnvironmentalVariable - for example:

{
  "schemaPath": "...",
  "env": {
    "production": {
      "appProd": {
        // BEWARE OF THE BELOW PATTERN!
        "env": { ... }
      }
    }
  }
}

Now the reasoning for it: let's start with a most simple use case - one app/one schema.

Single-app/single-schema

Usually a single app requires a schema path/url and an env variable to facilitate the build and deployment:

export type GraphQLAppConfig = {
  schemaPath: FilePath,
  schemaUrl: URL,
  env: EnvironmentVariable
};

export type EnvironmentVariable = {
  [envVarName: string]: GraphQLAppConfig
};

Multiple-apps/isolated directories

For multiple apps with isolated directories, I like @wincent's idea about a common-default GraphQL configurations, but in spirit of keeping things simple at first, I'd like to propose an one-config-per-directory approach as our first draft. I think there's an edge case where we'd have multiple parent GraphQL configurations to consider:

./repo/.graphqlrc
./repo/appFamily/.graphqlrc
./repo/appFamily/app1/.graphqlrc

This can become a discussion of its own, which we can have separately ;)

Custom validation rules

You may want to run custom validation rules before building GraphQL apps/codegen/etc - customValidationRules is useful for that.

Reserved namespaces

@wincent made a suggestion to provide a way to include any other configurations if you desire. This namespace will contain an app/prod/organization/team-specific GraphQL configurations, and it will be treated as optional settings you'd like to use for your purpose.

{
  "schemaPath": "path/to/schema",
  "extensions": {
    "my-tool": {
      "here": "This is a tool-specific extension"
    }
  }
}

What is this repo, then?

I think this repo should serve two roles in a broader sense:

1. Maintain this protocol and become a town hall for discussions/improvements for this protocol.
2. Provide reference implementations of helper methods to lubricate adoption for this protocol.

An example of the reference implementation of the helper method: we mentioned a way to find this GraphQL configuration is to walk up a directory until it finds one. This basically can be implemented as such:

/**
 * (From `graphql-language-service` repo:
 * Finds a .graphqlrc configuration file, and returns null if not found.
 * If the file isn't present in the provided directory path, walk up the
 * directory tree until the file is found or it reaches the root directory.
 */
export function findGraphQLConfigDir(dirPath: string): ?string {
  let currentPath = path.resolve(dirPath);
  while (true) {
    const filePath = path.join(currentPath, '.graphqlrc');
    if (fs.existsSync(filePath)) {
      break;
    }
    if (isRootDir(currentPath)) {
      break;
    }
    currentPath = path.dirname(currentPath);
  }
  return !isRootDir(currentPath) ? currentPath : null;
}

function isRootDIr(path: string): boolean {
  return path.dirname(path) === path;
}

As there are several use cases, we can tackle each of them and present a way to use this configuration programmatically. I can begin by identifying/suggesting some of them below:

• A generalized module bundler configurations, using GraphQL configurations
  • webpack, yeoman, babel, and etc...
• How to include custom validation rules in your application
• How to generate schema with provided schema path(s)
  • Sending introspection query to the endpoint
  • Parsing/building the schema using a local file path
  • Compiling GraphQLSchema object using schema definitions in .js
  • How to cache the schema
    • local files, indexedDB, and etc...

Action items

There are many things to do! But when we successfully deliver this, I believe we can get to the point where we have a shared GraphQL configuration that everyone agrees on, with commonly-used techniques and best practices to configure your GraphQL environment.

• Write a draft for this protocol
• Populate this repository with those helper methods
  • Discuss what the superset of those helper methods would be
  • Prioritize by the most impactful and common ones and implement them
  • Generalize existing implementations and move them here
• Update other remaining documentations, including this repo's README
• Suggest GraphQL configuration as a best-practice at graphql.org

Lastly, although we've begun this proposal and made a bunch of suggestions, I'd love to have a collaborative effort in pushing this forward. It'll be amazing to see us working together to improve and deliver this proposal to the GraphQL community.

Please post your thoughts/concerns/questions! Also if you'd like to be responsible for one of the actions items above, don't be hesitate to do so :D

[nodkz]

@asiandrummer very clean! I'd like it.

1. Small fixes by schema definition:

- env?: EnvironmentVariable,
+ env?: GraphQLConfigurationEnvs, 
- extension?: Any,
+ extension?: GraphQLConfigurationExtension,   

+ export type GraphQLConfigurationExtension = {
+  [toolName: string]: any,
+ };

2. And where I should add schemaEntrypoint path to the schema js file which exports GraphQLSchema (in the root or under extension key)? Of course, I would like to see this common config key along with schemaPath and schemaUrl.

[asiandrummer]

@nodkz edited with suggestions! For 2., I added schemaJS to specify the schema constructed with graphql-js.

[nodkz]

@asiandrummer schemaJS is not a better name if we want to keep .graphqlrc agnostic to programming language. I agree that schemaEntrypoint also not so good as wanted.

[asiandrummer]

@nodkz I'm also terrible at naming things ;) I'm open to other suggestions please!
Also, I propose .graphqlconfig instead of .graphqlrc.

[asiandrummer]

Another addition that may come later is a glob pattern for locating directories:

{
  "inputDirs": [ "**/apps" ],
}

[asiandrummer]

@nodkz @schickling I'm going to write something up to be included in graphql.org page soon - I'll cc both of you to the PR/issue in that repo when I create one!

[asiandrummer]

Progress update: I haven't spent much time to work on this until today - but I still wanted to communicate how I thought the further process should be.

1. Write up the best practices proposal to graphql.org (@asiandrummer)
2. Implement/move over the wrapper/GraphQLConfig code to this repo (up for grabs)
3. Open pull requests to the major GraphQL libraries to integrate the proposed GraphQL configuration (continuous effort from everyone!)

I'm in progress of completing (1), and will open a PR once done and include stakeholders. The whole progress has been somewhat stagnant and I apologize for the delay, and I'll make sure to spend time to work on this consistently.

For that, I have some sort of deadline in my mind (by the end of March) to finish this project.

[schickling]

Thanks a lot for that great progress update. We're happy to tackle (2). Should be feasible within the next week. I'll ping you in Slack to coordinate on this.

[mjmahone]

I'm trying to wrap my head around what the inputs/outputs of something utilizing this config is supposed to be.

I think it's

• input: some schema file + files containing GraphQL definitions
• output: a validated base schema, parsed GraphQL AST, and maybe a schema that includes any extensions from the definitions files.

One of the problems I've run into, is that I need to parse graphql from a lot of different sources: JS files, .graphql files, schema files, etc., and the AST coming out of each individual source is often not able to be validated until combined with all the other AST sources. As such, I think GraphQLConfiguration should support combining files that need to be parsed in entirely different ways, apply a schema to the parsed result, so you can get a validated schema + definitions.

Some tweaks I'd propose making:

• Split the internals of GraphQLConfiguration into a ParsingConfiguration and SchemaConfiguration.
• A GraphQLConfiguration is then a union of a list of ParsingConfigurations, a SchemaConfiguration, and validation rules.

Additionally, it seems like env should always live outside, not inside, the configuration. Each configuration describes a single environment (whether that's production or development or separate apps). When parsing raw files, you can choose to use multiple configurations, and switch to the relevant configuration (you might have a map of map of configurations, if you have 3 different apps to configure, each with their own environments. But that should be fine).

[asiandrummer]

@mjmahone

As such, I think GraphQLConfiguration should support combining files that need to be parsed in entirely different ways, apply a schema to the parsed result, so you can get a validated schema + definitions.

Wouldn't we be still able to use extensions and includeDirs (I renamed it from inputDirs) to do this? The different file types could be determined with the extension name, and then it'll be a matter of differentiating the parsing method after consuming the configuration file.

re: env variables

The env variables could certainly be used in that way, but I could see how this could be useful if, during the build/compile time, the tool used wants to choose a different environment. I prefer to separate those using app configs though

[asiandrummer]

cc @nodkz @schickling @mjmahone @wincent @martijnwalraven - including everyone who participated in the discussions for a final review.

I've taken some time to get more feedbacks internally from Facebook, iterate on the writing a few times, and drafted up a final version of what I'll create an issue with Monday to graphql.org. I'd appreciate if you could take a look - we can also continue the discussion in the issue I'll be creating.

The most notable changes are the addition of projects property and introducing the "experimental
configuration options" field. The previous draft was comprehensive in supporting various use cases, but 1) it included a possible recursive pattern in properties like env that made the configuration more complex than it should be, and 2) it included properties that aren't supported in different codebases, and I think adoption will be easier if we started with minimal configurations and iterate on it to support more use cases.

For the reason above, I've moved env property to "experimental configuration options" - @nodkz I'd love to chat with you periodically to be able to include this as one of the "official recommendations". Until then, I propose we keep testing env property by having it in the extensions property, and with some more use cases I hope we can include in the official recommendation.

Lastly, for the starters, I'll be submitting a few PRs to make graphql-config a place for util functions and/or the reference implementations of them. They won't be as comprehensive as we'd like them to be, but hopefully with the additions for those util functions we'd be able to claim this repository the place for them.

GraphQL Configuration

There are many ways to configure your application to use GraphQL, and while it is often enough to specify configuration options directly in your application code, maintaining and understanding the hard-coded configuration options may become a challenge as the scale grows. We recommend configuring your application with a .graphqlconfig file that contains commonly needed GraphQL-related artifacts.

GraphQL Configuration Format

After inspecting many GraphQL applications for use cases and communicating with many external contributors, we recommend configuring your GraphQL application with a below format:

// For the ease of understanding/formatting, we're using a
// Flow-like type to define the configuration format.

type GraphQLConfiguration =
  GraphQLProjectConfiguration & {
    projects?: {
      [projectName: string]: GraphQLProjectConfiguration
    }
  };

type GraphQLProjectConfiguration = {
  schemaPath?: string, // a file with schema IDL
  schemaUrl?: URL,

  // For multiple applications with overlapping files,
  // these configuration options may be helpful
  includeDirs?: Array<DirPath>,
  excludeDirs?: Array<DirPath>,

  // If you'd like to specify any other configurations,
  // we provide a reserved namespace for it
  extensions?: {[name: string]: GraphQLConfigurationExtension}
};

type GraphQLConfigurationExtension = {
  [name: string]: mixed
};

Use Cases

Usually, for a simple GraphQL applications, the only required configuration is a way to fetch a GraphQL schema:

{
  "schemaPath": "./schema.graphql"
}

Although it is possible for the configuration file to have more than one way to fetch the schema, a GraphQL application consuming this configuration should determine how to work with provided configuration properties. For example, while one application may arbitrarily choose to have schemaPath take precedence over schemaUrl, another may choose to throw and error when both are provided. The behavior in this case is explicitly undefined.

{
  // How to resolve this is up to the application
  "schemaPath": "./schema.graphql",
  "schemaUrl": "http://my-app.com/schema"
}

For multiple apps with isolated directories, there are mainly two ways to set up the configuration. Each app can either use the projects property to specify each application's configuration, or have a separate .graphqlconfig file for each project root.

{
  "projects": {
    "appA": {
      "schemaPath": "./appA/appASchema.graphql"
    },
    "appB": {
      "schemaPath": "./appB/resources/appBSchema.graphql"
    }
  }
}

Or each can app have a separate .graphqlconfig file per each application and/or directories:

./appA/.graphqlconfig
./appB/.graphqlconfig

Default Configuration Properties

Treat the top-level configuration properties as default values to use if a project configuration scope omits them.

Consider the below GraphQL configuration:

{
  {
    "projects": {
      "projectA": {
        "schemaPath": "./resources/schema.graphql",
        "includeDirs": "./projectA/graphql"
      },
      "projectB": {
        "schemaPath": "../resources/schema.graphql",
        "includeDirs": "./projectB/graphql"
      },
      "projectC": {
        "schemaPath": "./schema-for-projectC.graphql"      }
    }
  }
}

Since projectA and projectB share the same schema file, we can push the schemaPath property to the top level, and treat it as a default value for a schema path. In this case, the application using this configuration should treat each project config as a more specific scope, and look at the top level scope with default properties as a fallback.

{
  "schemaPath": "./resources/schema.graphql",
  {
    "projects": {
      "projectA": {
        "includeDirs": "./projectA/graphql"
      },
      "projectB": {
        "includeDirs": "./projectB/graphql"
      },
      "projectC": {
        // "schemaPath" in this scope should take precedence
        "schemaPath": "./schema-for-projectC.graphql"
      }
    }
  }
}

Extensions as Namespaces

If your application requires a unique configuration option, use the extensions attribute to customize your own configuration option.

Additionally, we'd like to treat the extensions property as a sandbox for improving the overall GraphQL configuration. If there is a configuration option that you think will be useful for general purposes, please let us know! Meanwhile, take advantage of this 'extensions' for testing purposes.

For example, some application may choose to build using Webpack and need to specify Webpack-related configurations in GraphQL configuration. Also, they may need to process additional file types other than .graphql. Below suggests one way to configure these options using 'extensions':

{
  "schemaPath": "...",
  "extensions": {
    "webpack": {
      // Webpack-specific options here!
    },
    // additional file types that you want to process
    "additionalFileTypes": [ '.js' ]
  }
}

Experimental Configuration Options

Below are what we're experimenting with for additional fields. Before including in the official recommendation, we would like to iterate on a few different codebases first.

'Env' Variable

An env property is useful to facilitate the build and deployment if your application needs to introduce a GraphQL-specific environment variable and run build steps with it:

// run a development build with a production GraphQL database 
GRAPHQL_ENV=production && NODE_ENV=development && babel-node ./server.js

Also, an env property allows an easier adoption of this GraphQL configuration format for non-JavaScript codebases.

The proposed usage for the property is as follows:

{
  "schemaPath": "./schema.graphql",
  "env": {
    "production": {
      "schemaUrl": "http://your-app.com/graphqlschema"
    },
    "development": {
      "schemaPath": "./dev-schema.graphql"
    }
  }
}

GraphQL configurations do not support this property because there isn't a clear example how to utilize this property yet.

Configuring a schema defined programmatically

Suppose you have a schema defined using objects from graphql-js:

// In schema.js,
const queryType = new GraphQLObjectType({ name: 'Query' });
export const schema = new GraphQLSchema({ query: queryType });

Because this is a valid GraphQL schema that can be used within applications, we want to provide a way to configure the schema defined in this way:

{
  "schemaModule": "./schema.js"
}

This is not yet supported in GraphQL configuration because we're not sure what shape the type for the exported module is, nor whether this belongs in a JS-specific sub-configuration.

Including a set of customized validation rules

Sometimes a GraphQL application wants to either define an additional set of validation rules or specify a list of validation rules to be run at build/runtime. The GraphQL configuration can be a good place to locate where to look for this:

// In customRules.js,
export const customRules: Array<(context: ValidationContext) => any> = { ... };

// And in .graphqlconfig, specify where the custom rules are:
{
  "customValidationRules": "./customRules.js"
}
{
  "customValidationRules": [ "./customRules.js", "./moreCustomRules.js" ]
}

We're not yet officially supporting these rules, because many GraphQL tools don't yet have a way of using additional validation rules.

Helper/example libraries

To illustrate the GraphQL configuration in code as well as the behaviors defined in this documentation, we've collaborated to provide a few helper libraries for GraphQL configuration:

• https://github.com/graphcool/graphql-config
  • TODO: explain the util functions in this repo and how they work
  • TODO: actually PR out some util functions
• https://github.com/graphql/graphql-language-service

[nodkz]

@asiandrummer sorry i'm on vacation now till 18 April.
After reading from mobile at first look all is okay. But it is too difficult make good review via phone.

Btw, for me projects and env are the same (they have different nature but same config definition). So for config file simplicity you may remove env, I can use projects as env 😉

[RomanHotsiy]

Hey @asiandrummer, @nodkz, @schickling,

@IvanGoncharov and I started working on the initial implementation of this proposal on the branch of this repo. After reviewing this and previous threads we have a few suggestions/proposals.

First of all, I want to emphisize an important use case: the ability to execute GraphQL queries from inside the editor. I don't see so much value in writing GraphQL queries inside editor if you can't execute them from inside. In such case, users will just write queries in GraphiQL and copy-paste them to editors. And since GraphiQL already has validation and auto-suggestions, there is no need for those in editors. E.g. GraphQL Intelliji plugin supports this as a feature:

Execute queries with variables against configurable endpoints

So I think this is really important use-case which we should support in graphql-config core.

An env property is useful to facilitate the build and deployment if your application needs to introduce a GraphQL-specific environment variable and run build steps with it:
GraphQL configurations do not support this property because there isn't a clear example how to utilize this property yet.

The example of utilizing this property is GraphQL Intelliji plugin which has its own config format at the moment. This plugin has built-in support for executing GraphQL queries and there is config property endpoints which can be replaced by env.

Also, we think schemaUrl should be renamed to serverUrl. This way it is clearer that this is GraphQL server, not just URL which dumps introspection JSON.

Although I think this is a good example for each app to customize their settings, I'm not sure if we should recommend the custom HTTP headers in a .graphqlrc file. I think at the time when .graphqlrc configurations are needed, users should already have been authenticated to use it. Obviously, I'm only thinking a couple of use cases here and would love to hear some other cases where including auth options in .graphqlrc would help a lot.

There is an important use case: writing project against public APIs like GitHub, Yelp, Shopify. In such case, users can't be already authenticated because of obvious reasons.
The issue of storing secure information in config can be solved by using env variables:

type GraphQLHeaders = {
  [name: string]: string | { envVariable: string }
}

Summing up, we think it makes sense to move server url and headers to core config. In Flow-like it may look like:

// For the ease of understanding/formatting, we're using a
// Flow-like type to define the configuration format.

type GraphQLConfiguration =
  GraphQLProjectConfiguration & {
    projects?: {
      [projectName: string]: GraphQLProjectConfiguration
    }
  };

type GraphQLProjectConfiguration = {
  schemaPath?: string, // a file with schema IDL
  server?: ServerConfig,

  // For multiple applications with overlapping files,
  // these configuration options may be helpful
  includeDirs?: Array<DirPath>,
  excludeDirs?: Array<DirPath>,

  // If you'd like to specify any other configurations,
  // we provide a reserved namespace for it
  extensions?: {[name: string]: GraphQLConfigurationExtension}
};

type ServerConfig = {
  url: URL,
  method: string,
  headers?: GraphQLHeaders
}

type GraphQLHeaders = {
  [name: string]: string | { envVariable: string }
}

type GraphQLConfigurationExtension = {
  [name: string]: mixed
};

What do you think?

About the idea suggested by @wincent:

Also note another idea here: that we can set common defaults in .graphqlrc higher up in the hierarchy (at the repo root) and have app-specific overrides in the app-local dirs.

This is a great idea but it introduces difficulties. If merge configs from different levels we loose info about config location and are not able to resolve relative paths from config. So we suggest to not merge configs, at least in the initial version.

[IvanGoncharov]

Together with @RomanGotsiy we implemented @asiandrummer proposal on the config-protocol branch. We borrowed a lot of ideas from graphql-language-service-config since it provides everything you need for GraphQL support inside IDEs by having per-file API.

However, not every tool works on per file basis, e.g. mocking your GraphQL API or checking schema for breaking changes. So we tried to accommodate both cases by splitting functionality into the two classes:

class GraphQLConfig {
  public config: GraphQLConfigData
  public configPath: string

  constructor(
    public rootPath: string = process.cwd()
  )

  getProjectConfig(projectName: string): GraphQLProjectConfig
  getConfigForFile(filePath: string): GraphQLProjectConfig | null
  getProjects(): { [name: string]: GraphQLProjectConfig }
}

class GraphQLProjectConfig {
  public config: GraphQLProjectConfigData
  public configPath: string

  constructor(
    path: string = process.cwd(),
    public projectName: string = process.env.GRAPHQL_PROJECT,
    configData?: GraphQLConfigData // for cases when data is already parsed
  )

  resolveConfigPath(relativePath: string): string
  resolveSchema(env?: string): Promise<GraphQLSchema>
  resolveIntrospection(env?: string): Promise<any>
  resolveSchemaIDL(env?: string): Promise<string>
  getConfig(envName: string = process.env.GRAPHQL_ENV): GraphQLResolvedConfigData
  getEnvs(): { [env: string]: GraphQLResolvedConfigData }
  includesFile(filePath: string): boolean
}

The simplest example would be:

  const config = new GraphQLProjectConfig('./', 'OptionalProjectName');
  const schema = config.resolveSchema('OptionalEnvName');

In more complicated scenarios you can get per-file schema like that:

  const config = new GraphQLConfig('./');
  const perFileConfig = config.getConfigForFile('./someQuery.js');
  const perFileSchema = perFileConfig.resolveSchema('OptionalEnvName');

@asiandrummer, @nodkz, @schickling,
What do you think about proposed API?

[asiandrummer]

@IvanGoncharov @RomanGotsiy first of all thanks for taking point to implement the details! Please tag me in the PR once you make one and I'll make sure I review it. But before anything I've read through your proposals and wanted to ask some questions below.

@IvanGoncharov from your proposed API, I don't think env should be used as the main parameter for functions to resolve which projects the config is dealing with - we have projects for that (in graphql-language-service-config I think I'm calling it an appName).

I don't see so much value in writing GraphQL queries inside editor if you can't execute them from inside.

I disagree - although being able to execute queries inside the editor is a cool feature addition to each IDEs, it doesn't devalue the ability to run the language service in each IDEs. For the simplest example, the query validation would serve as the first line of defense in passing your queries as usable ones. Go-to definition support is almost completely lacking in GraphiQL, and is a different use case from executing queries.

env variables

I agree that's a clearer use case - however I'd be more comfortable to add it in the actual spec doc (if we get to have one later) after seeing some more use cases/needs. The main concern I have is that the env variable doesn't directly influence setting up the GraphQL project configs, rather it does so for the build/compile steps for GraphQL. My argument is that we can include each environments as each projects, but I can see how env variable is more familiar for some users.

The issue of storing secure information in config can be solved by using env variables:

Sorry if I'm misunderstanding, but could you elaborate here? How could you work around storing a sensitive information in a JSON file by using env variables, which would be stored in the same JSON file?

Summing up, we think it makes sense to move server url and headers to core config.

I'm not sure why more than a server URL is needed to configure the GraphQL server, even if you're using the server information from the config to do language-agnostic stuffs (such as build/compile). Considering the mentioned use case is being able to authenticate and fetch information easily, I can see why putting the server configuration in the JSON file (graphqlconfig) is more convenient - if one chooses to do so, extensions is available to use.

My point is that it feels like the detailed server information/options in GraphQL configuration file won't really be a general configuration option (and personally a discouraged approach). I would like to understand more about the needs for such an info since I might be missing something huge here.

[pranaygp]

An addition I'm hoping to add is a name field. It would make the type as so:

type GraphQLProjectConfiguration = {
  name?: string
  ...

At Facebook, I'd like to have this for a script to uniquely identify individual projects that exist throughout our codebase. While in my case, I require each name to be unique, that doesn't have to be a part of the spec. This can simply by a useful tool for existing tools out there to use the name value to refer to the project when outputting debug messages. (within https://github.com/graphcool/graphql-config/blob/config-protocol/src/utils.ts#L56 , we could use this for nice error messages 😄 )

For multi-project, this might not be as useful since a projectName is already defined as keys, but might still be useful for human friendly names?

For other scripts and tools that require a way to uniquely identify projects (like I do), this could serve that purpose 😃

[IvanGoncharov]

from your proposed API, I don’t think env should be used as the main parameter for functions to resolve which projects the config is dealing with - we have projects for that (in graphql-language-service-config I think I’m calling it an appName).

@asiandrummer We made implementation according to your proposal so we fully support projects. Additionally, we implemented 'Env' Variable from Experimental Configuration Options section of your proposal and envparameter is used to choose one of environments. If env section is not specified in config, env parameter should not be provided.

The issue of storing secure information in config can be solved by using env variables:

Sorry if I’m misunderstanding, but could you elaborate here? How could you work around storing a sensitive information in a JSON file by using env variables, which would be stored in the same JSON file?

A @RomanGotsiy example was about OS environment variables not environment variable inside config file. Our idea is to allow you to specify header both as a string or as a reference to OS environment variable.
For example, GitHub GraphQL API requires you to specify User-Agent and Authorization headers:

{
   "headers": {
       "User-Agent": "Build script",
       "Authorization": {
           "envVariable": "GITHUB_AUTHORIZATION"
       }
   }
}

In above sample User-Agent is specified inline but Authorization references GITHUB_AUTHORIZATION environment variable you need to setup in your OS environment.

My point is that it feels like the detailed server information/options in GraphQL configuration file won’t really be a general configuration option (and personally a discouraged approach). I would like to understand more about the needs for such an info since I might be missing something huge here.

At Facebook, you work only with your internal GraphQL APIs which are accessed behind a firewall so you don’t need any Authorization to run introspection query. However, at some point, you may allow 3rd-party developers to access Facebook GraphQL API and I’m sure in order to run queries those 3rd-party developers will be required to pass some kind of header even for introspection query.

And this is not theoretical. GitHub, Shopify, Yelp and others already have public GraphQL APIs that require HTTP headers in order to run any query including introspection one. If we don’t support headers, how 3rd-party developers can write a config that gets introspection from those APIs?

BTW. I’ve just watched Robert Zhu talk at the OpenStack conference and here is a relevant slide from it:

I fully agree that you need to handle Authorization as part of business logic. However, Authentication according to this slide should be handled before GraphQL layer that means you need to authenticate API client even for the introspection query and HTTP header is one of the most popular Authentication methods for Web APIs.

---

@pranaygp Thank you for taking a time to review our implementation.

This can simply by a useful tool for existing tools out there to use the name value to refer to the project when outputting debug messages.

Can the file path to config file be used to uniquely identify config inside error messages?
In both graphql-language-service-config and in our implementation, if a lib doesn’t find the config file in the current directory it searches up the directory tree. In such case it would be hard to locate invalid config by the name property inside error message.

[asiandrummer]

Additionally, we implemented 'Env' Variable from Experimental Configuration Options

Cool - sorry for the confusion. I missed that one.

you work only with your internal GraphQL APIs which are accessed behind a firewall

On the contrary, we do have some internal usages that require access tokens to reach GraphQL endpoint. Just like the slide you've included, the setup to use the endpoint is done within the code, and the necessary configuration is included close to the code that it's used to execute the http request.

you need to authenticate API client even for the introspection query and HTTP header is one of the most popular Authentication methods for Web APIs.
How 3rd-party developers can write a config that gets introspection from those APIs?

I'm fully in agreement with you that most of the times a proper authentication is needed to access third-party GraphQL endpoints, including the data points you've mentioned. What I want to reiterate is that GraphQL configuration may not be the right place for them. I want to accentuate the need for the separation of concerns - to summarize, the core of GraphQL configuration should only achieve a few things: defining the location to get GraphQL schema, and defining where the relevant files are to GraphQL artifacts. In addition, projects and extension fields can help to provide a necessary extensibility for different organizations to suit their needs. On the other hand, I don't believe a server configuration is a necessity for configuring GraphQL projects - there are many other ways to set the server-side configurations, but I don't think it'll serve as the core configuration options.

[asiandrummer]

Also, thanks to everyone's participations, I feel like we're ready to move this forward and actually create a PR to add this as a protocol/spec document. I'll get that done soon - we can continue these valuable conversations over there.

[IvanGoncharov]

@asiandrummer Just to be sure we are on the same page here, where should schemaUrl point to? URL that has IDL/JSON or URL to GraphQL server and you send introspection query to get the schema?

Because in my and @RomanGotsiy comments we assumed that schemaUrl is URL to GraphQL server. There is no definition of this field inside your proposal so the only clue is the following example from you proposal:

{
  // How to resolve this is up to the application
  "schemaPath": "./schema.graphql",
  "schemaUrl": "http://my-app.com/schema"
}

[schickling]

Hi @IvanGoncharov, apologies for the radio silence on my end these days. I was meeting with @asiandrummer (and other amazing folks from Facebook) this week and we discussed the next steps for graphql-config. (I also discussed this with @jbaxleyiii from Apollo.)

We concluded the following:

• schemaPath will be a required configuration option and the only source of truth for the schema. (Both SDL (formerly IDL) and JSON introspection results are supported as source)
• The schemaUrl option will be replaced by another workflow. Things like HTTP headers etc will be handled by this CLI tool.
• There should be so called "standard extensions" that ship with helper functions in graphql-config that other libraries can use (e.g. http endpoint, subscription endpoint)

To discuss:

1. Define naming conventions for extensions
2. Define standard extensions
3. File format (JSON, YAML, HCL, ...)
4. Default value for includeDirs & excludeDirs & agree on regex format

Examples

To wrap up, here are a few examples using the current format proposal.

Minimal

{
  "schemaPath": "./schema.graphql",
  "extension": {
    "endpoint": "https://api.graph.cool/instagram"
  }
}

Multi-project

{
  "development": {
    "schemaPath": "./dev.graphql",
    "extension": {
      "http-endpoint": "http://localhost:3000/graphql"
    }
  },
  "production": {
    "schemaPath": "./prod.graphql",
    "extension": {
      "http-endpoint": "https://api.graph.cool/instagram"
    }
  }
}

[IvanGoncharov]

We have some questions about projects feature since there are a couple very distinct use cases to be solved:

1. You have a multi-projects repo where you have different folders tied to different graphql-schemes.
2. You have singe-project repo but you have different schemas for production and testing setups.
3. The combination of 1 and 2. So you have a multi-project repo where each project has multiple setups like testing and production

You previously mentioned that env can be replaced with projects:

My argument is that we can include each environments as each projects

But there is an issue with that. Assuming use case 2 and 3 and e.g. the following config:

{
  "production": {
     "schemaPath": "./prod.graphql",
  },
  "dev": {
     "schemaPath": "./dev.graphql",
  }
}

With the current language services API:

getAppConfigNameByFilePath(filePath: Uri): ?string {

how would you decide which schema to use? As far as I understand the current use case is editor passes path to the file and that's it.

One option would be changing getAppConfigNameByFilePath to return an array of matching project names. But we think it's better to have two different mechanisms for different use cases:

• projects for matching file to a schema (with includeDir/excludeDir)
• env for matching execution environment to a schema

What do you think?

[asiandrummer]

@IvanGoncharov allow me to reply to your comment in regards to schemaUrl tomorrow after I get some sleep please ;)

For all 3 cases above, if you cannot associate one project with one schema, with a list of includeDirs and excludeDirs, or in the case like 3 where we may have a multiple development environments, the extensions could probably be very useful:

{
  "projects": {
    "project1": {
      "extensions": {
        "env": {
          "production": { ... }
        }
      }
    }
  }
}

I chatted briefly with @schickling about supporting features in extensions with helper functions - I think it'd be a good idea to leave the env field as optional "experimental" feature, but I'd agree to implement using env field from the graphql-config helper library since it sounds like one of the most asked features.

Would this be okay?

[IvanGoncharov]

allow me to reply to your comment in regards to schemaUrl tomorrow after I get some sleep please ;)

@asiandrummer Good night 😄 No need to reply. It's now clear from @schickling comment.

Would this be okay?

Yes, it would be a good starting point.

[IvanGoncharov]

@asiandrummer together with @RomanGotsiy and @schickling we've discussed env situation once more time and agreed that the only use case we can come up is describing GraphQL endpoints.

@nodkz Presented another use case for env but it can be also solved by a projects:

Btw, for me projects and env are the same (they have different nature but same config definition). So for config file simplicity you may remove env, I can use projects as env 😉

That's why we decided to remove env and instead attach names directly to endpoints, so the end result looks like this:

{
  "schemaPath": "./schema.graphql",
  "extensions: {
    "endpoint": {
      "dev": {
        "url": "http://localhost:3000/graphql",
        "subscription": {
          "url": "ws://localhost:3001"
        }
      },
      "prod": {
        "url": "https://your-app.com/graphql",
        "headers": {
          "User-Agent": "Build script",
          "Authorization": "bearer ${env: YOUR_APP_TOKEN}"
        },
        "subscription": {
          "url": "wss://subscriptions.graph.cool/v1/instagram"
        }
      }
    }
  }
}

We also made a few small changes to your proposal as you can see in this diff.
Here is the short summary:

• remove schemaUrl per @schickling comment
• .graphqlconfig => .graphqlrc since .*rc is almost universal pattern for naming config files
• use Glob patterns inside include/exclude
• add endpoint extension

What do you think?
How should we proceed further? Should we create a PR to add spec.md to the master?

[nodkz]

@IvanGoncharov I prefer env name instead projects due following reasons:

• obtain config file which will be the same shape with .babelrc. In future, it allows more seamlessly integrate different tools. In current understand nature of configs and how it works (i do not want one more standard).
• in Docker, CI and other devOps tools you setup ENV_VARIABLES for running the application in development, production staging or testing modes. So env is more suitable name for more devOps than projects.

As a working example of complex .babelrc.js with bunch of envs for my isomorphic app can be found here. Babel config shape which can be borrowed:

{
  ...defaultOptions,
  env: {
    production: {
      ...mergableOptionsWithDefaultForProduction,
    },
    development: {
       ...mergableOptionsWithDefaultForDevelopment,
    },
    staging: {
       ...mergableOptionsWithDefaultForStaging,
    },
    [anyOtherEnvName: string]: { ...opts },
    // projects can be setupped in following manner
    production_projectA: { ... },
    production_projectB: { ... },
  },
};

PS. @asiandrummer how do you setup .babelrc for your multiple-apps at facebook? How do you think it's possible to write graphql configs in the same manner like existed babel configs in your mono-repo?

[IvanGoncharov]

in Docker, CI and other devOps tools you setup ENV_VARIABLES for running the application in development, production staging or testing modes.

@nodkz Biggest problem I see with environment variables is that they are CLI oriented. One of the main use cases for graphql-config is to provide a schema to GraphQL plugins for popular editors(IntelliJ, Atom, etc.).

And this is where configuration through environment variables create a few problems:

1. Many developers start editor by clicking an icon and not by running CLI command so they don't inherit environment variables.
2. Even in case if you always run editor from CLI. Editor instances are long-lived, especially if you have multiple tabs so they will not react to changes of env variable and you will be forced to restart the editor.

So by using env as a name, we suggest that primary way to switch them are environment variables which as I showed above doesn't play very well with GUI editors. On the other hand, projects using include/exclude are used to distinguish which schemaPath to use on a per-file basis without a need for environment variables.

[nodkz]

I'm using Atom by clicking an icon and not by running CLI command. Atom makes eslint checks which use eslint-import-resolver-babel-module plugin which reads configs of plugin babel-plugin-module-resolver from .babelrc. So internally for my complex setups I do not use any env. No need to start editors from CLI with env variables.

From babel docs:

The env key will be taken from process.env.BABEL_ENV, when this is not available then it uses process.env.NODE_ENV if even that is not available then it defaults to "development".

So for IDE by default it tries to get merge(configFile.env.development.devOpts, configFile.defaultOpts)

{
   ...defaultOpts,
   development: {
     ...devOpts,
   }
}

if development is absent it takes options from configFile.defaultOpts.

And yep, it is cool feature to run editor with ENV variable for obtaining production/staging transpilling, checks and connections.

---

I have not seen in any .dot/.rc/.config file a mention of project. And it does not meet with YAGNI.
projects solved in babel by placing in every folder (which you consider as a project root) dedicated .babelrc file. .eslintrc works in the same way - you may put on every folder/subfolder .eslintrc file and it will be merged with files on top levels. As a result you take very customizable configs with inheritance.

projects also does not play very well with GUI editors. You should use it in conjunction with include/exclude and some errors in configs will be quite hard to debug if some files/folders are overlaps. If include/exclude is planned to be very simple, so why not to put in every include/exclude folder its own .graphqlconfig file?

How I understood projects with include/exclude only needs to the current facebook's multiapp structure. And I think that this part can be solved without this complex, weird and absolutely new structure in config files. Anybody else needs projects option? Or we can make it optional (and it will be implemented only in Nuclide)?

[schickling]

@asiandrummer I assume you're understanding Facebook's use case best, can you provide some background why this is desirable? Having multiple "smaller" .graphqlrc files would be nicer instead of one big multi-project config file.

[asiandrummer]

@nodkz @IvanGoncharov @schickling Let me come back in a couple hours and answer this if that's okay with you guys - sorry! Things are a bit crazy on my side right now.

[asiandrummer]

IMO I'd prefer .graphqlconfig as it's clearer to me that it's a file that stores configuration for GraphQL. And glob patterns for include/excludeDirs sounds good to me - I actually forgot to add that in! endpoint sounds like a great example for extensions.

With the discussion around Babel configuration files, I actually think babelrc and graphqlconfig serve two almost completely different use cases, which I can elaborate below. But for this reason I don't think we can relate the usages and purposes for .babelrc and .graphqlconfig.

About .babelrc

Within Facebook we include .babelrc files wherever it's needed, and that works well because setting up a .babelrc is same as passing options to Babel for transforms, hence it's not always necessary to create and write the options into .babelrc file. I've seen some cases where the actual transform code contains the necessary options for transforms, especially when we're dealing with more than a couple project folders.

I would argue that configurations for Babel and GraphQL tools/apps have vastly different purposes. Babel is a tool that accomplishes a specific set of goals, thus the goal of a configuration file (.babelrc) is to help with that specific set of goals by providing JSON-serializable necessary options (including env option to specify environments) to Babel. GraphQL configuration already has more than one category of software to support - GraphQL tooling and GraphQL-powered apps. I believe the goal of .graphqlconfig is as follows:

• Provide the location for GraphQL schema
• Provide where GraphQL artifacts/files are

projects and env

IMO projects and env in .graphqlconfig would function exactly the same - it feels like that the difference is what each name implies. I just chose "projects" since "environment" suggests to me that the options inside it change the behavior of processes running with this configuration file, and that's not always the case.

@schickling I'm ready to move forward with the spec + some additions @IvanGoncharov made. Let me PR this out and we can continue this conversation in that PR.

[asiandrummer]

Closing as 9e7aef7 has been committed. I'll add endpoint extension suggestion to it