How to Store Operational API Configuration in Files

Operational API configuration refers to the set of non-sensitive values that define how an API behaves and identifies external resources across deployments. These values include port numbers, endpoint URLs, operation timeouts, log levels, connection limits, resource identifiers, and so on.

Factoring these values out of the code allows the same application build to be deployed across development, testing, staging, and production, keeping it independent of the deployment environment.

The short answer

The most common way to store operational configuration in files is to choose one representation and use it across every environment, while allowing each deployment to provide its own values.

As a general rule:

1. Choose the format according to the structure of the configuration.

   • Use key-value pairs for individual or mostly unrelated settings.
   • Use JSON or YAML for related or structured settings.

2. Use the same format and schema across environments.

3. Choose the file granularity according to how values change.

   • In the same file when configuration is used by a single service.
   • In separate files when configuration is shared between multiple services.

4. Choose the storage location according to who manages the configuration.

   • On your local machine for development and manual testing.
   • In a version-controlled repository or object storage for automated testing, staging, and production.

5. Keep the file outside the application build.

Choosing the configuration file format

Using key-value pairs

The most common format for configuration that consists mostly of independent scalar settings is a list of key-value pairs stored in a .env file.

For example:

SERVER_PORT=3000

LOG_LEVEL=info

RETRY_ATTEMPTS=3
RETRY_INITIAL_DELAY_MS=250
RETRY_MAXIMUM_DELAY_MS=5000

The main advantage of this format is that values can be loaded directly into the environment before the application starts and accessed in code using the process.env object.

The main drawbacks are that:

• Values are represented as strings that must be parsed and converted to their expected types before use (e.g., "3000" -> 3000).
• Relationships between settings are expressed only through their name prefixes (e.g., RETRY_ or DATABASE_).

Using JSON

The most common format for configuration that represents groups of related values or values with a meaningful hierarchy is JSON stored in a .json file.

For example:

{
  "server": {
    "port": 3000
  },
  "log": {
    "level": "info"
  },
  "retry": {
    "attempts": 3,
    "initialDelayMs": 250,
    "maximumDelayMs": 5000
  }
}

The main advantages are that:

• JSON allows configuration files to represent strings, numbers, booleans, arrays, objects, and null directly.
• JSON files can be directly imported as modules within the application.

The main drawback is that large JSON files can sometimes be cumbersome to write because of the extensive use of curly braces and commas.

Using YAML

YAML is a common alternative to JSON because it eliminates the visual noise of curly braces and commas.

For example:

server:
  port: 3000

log:
  level: info

retry:
  attempts: 3
  initialDelayMs: 250
  maximumDelayMs: 5000

The main advantage is that it allows developers to represent complex nested data structures while remaining easy to read and modify.

The main drawback is that it relies on whitespace indentation to define structure, so a single misplaced space or tab can break the file structure or silently alter how the configuration is parsed.

Keeping the file format consistent across environments

The configuration contract must remain completely consistent across deployments. An application should always access and handle its configuration in the exact same way regardless of the environment in which it runs. In practical terms, the application must not implement conditional runtime logic that couples configuration loading or parsing to a specific deployment target.

For example, an application should never load its configuration from a .json file in production while switching to a .env file for local development.

let config;

if (process.env.NODE_ENV === 'production') {
  const { default: productionConfiguration } = await import(
    './config/production.json',
    { with: { type: 'json' } }
  );

  config = productionConfiguration;
} else {
  config = process.env;
}

Choosing configuration file granularity

As a rule of thumb, always store configuration in a single file first, then split it by concern only when parts of it need to be reused, managed independently by separate teams, or mapped directly to modular configuration structures in your orchestration tools.

Single file configuration

In general, an environment's operational configuration should be stored in a single file by default. This allows developers to manage, version, and deploy the configuration as one unit, while allowing the application to load it without additional merging, ordering, or precedence rules.

For example:

{
  "server": {
    "port": 3000,
    "requestTimeoutMs": 5000
  },
  "database": {
    "host": "database.internal",
    "port": 3306,
    "poolSize": 10
  },
  "integrations": {
    "payments": {
      "baseUrl": "https://payments.internal",
      "timeoutMs": 3000
    }
  },
  "observability": {
    "logLevel": "info",
    "metricsEnabled": true
  }
}

The main drawback is that this approach doesn't promote the reusability of settings across multiple services, and it can lead to a bloated, monolithic file where operational boundaries become blurred.

Multi-file configuration

Splitting configuration into multiple files is mostly useful when part of it needs to be reused across multiple services, allowing shared values to be maintained in one place instead of being duplicated. When doing so, configuration should be split by top-level concern, such as server, database, integrations, observability, and so on.

For example:

{
  "server": {
    "port": 3000,
    "requestTimeoutMs": 5000
  }
}

{
  "database": {
    "host": "database.internal",
    "port": 3306,
    "poolSize": 10
  }
}

However, this approach implies managing additional files that can go missing or out of sync during deployment, as well as additional loading, validation, and merging logic inside the application. Furthermore, splitting configuration across multiple files introduces runtime risks, such as namespace collisions and deterministic ordering bugs, where the final operational behavior depends entirely on the sequence in which the code parses the files.

Naming and organizing configuration files

Configuration files should follow a predictable naming convention that makes both the target environment and the configuration scope immediately clear.

Single configuration files

The most common naming convention when an environment uses a single configuration file is to include the deployment name in the filename:

• .env.{deployment}
• {deployment}.json
• {deployment}.yaml

For example:

config
├── .env.development
├── .env.testing
├── .env.staging
└── .env.production

Multiple configuration files

The most common naming convention when configuration is split by concern is to use the deployment as the directory and the concern as the filename:

• config/{deployment}/.env.{concern}
• config/{deployment}/{concern}.json
• config/{deployment}/{concern}.yaml

For example:

config
├── development
│   ├── server.json
│   └── observability.json
└── production
    ├── server.json
    └── observability.json

Note that when splitting .env files by concern, you should maintain internal prefixes like SERVER_ or DATABASE_ to prevent naming collisions during runtime merging.

Choosing where to store configuration files

Across the software deployment lifecycle, operational configuration often shifts between three storage locations depending on whether the scope of the settings is local, private to a single service, or shared between several services.

The local filesystem

The local filesystem is generally used to store the development and testing configuration files. It allows developers to manipulate how the API behaves in isolation without triggering delivery pipelines or requiring peer reviews.

While this improves development speed and autonomy, it also introduces a higher risk of configuration drift if the changes are not documented or reflected into the staging and production files.

A version-controlled repository

A version-controlled repository (e.g., Git) is generally used to store the testing, staging, and production configuration files that are private to a specific service. It allows teams to review every change through a pull request, while enabling the CI/CD pipeline to automatically copy the file to the remote server during deployment.

While this provides a clear change history and keeps environments predictable, the main drawback is that updating even a minor parameter requires a code commit and a deployment pipeline run.

Object storage

Object storage (e.g., AWS S3, GCS) is generally used to store the staging and production configuration files that must be shared globally across multiple independent services. It allows developers to update shared configuration values independently of individual service repositories and deployment pipelines.

While this allows shared configuration to be maintained independently of individual service repositories and deployment pipelines, it also introduces a network dependency that can cause application boot failures if the cloud bucket becomes unavailable or its permissions are misconfigured.

Production considerations

The build phase should always produce a single, generic application artifact, such as a Docker image, that contains no environment-specific data.

Development                         Production

local config.json                   S3 object
       │                                │
       │                         deployment process
       │                                │
       └────────► config.json ◄─────────┘
                       │
                       ▼
                  Node.js API

To keep your build and release phases separate, your production configuration files must be stored entirely outside the build artifact.

The deployment process should combine the generic application artifact with the correct configuration file only when the application is released or started on the host server.

This allows you to use the exact same immutable image across development, testing, staging, and production by simply swapping the external file.