Skip to main content
Version: Next

Configuration

Configuration is a value object holding the SDK configuration. We can use it in two ways:

  1. When using Actor class, we can get the instance configuration via sdk.config 
    import { Actor } from 'apify';
    import { BasicCrawler } from 'crawlee';

    const sdk = new Actor({ token: '123' });
    console.log(sdk.config.get('token')); // '123'

    const crawler = new BasicCrawler({
        // ... crawler options
    }, sdk.config);
  2. To get the global configuration (singleton instance). It will respect the environment variables. 
    import { BasicCrawler, Configuration } from 'crawlee';

    // Get the global configuration
    const config = Configuration.getGlobalConfig();
    // Set the 'persistStateIntervalMillis' option
    // of global configuration to 30 seconds
    config.set('persistStateIntervalMillis', 30_000);

    // No need to pass the configuration to the crawler,
    // as it's using the global configuration by default
    const crawler = new BasicCrawler();

Supported Configuration Options

KeyEnvironment VariableDefault Value
memoryMbytesACTOR_MEMORY_MBYTES-
headlessAPIFY_HEADLESS-
persistStateIntervalMillisAPIFY_PERSIST_STATE_INTERVAL_MILLIS60e3
tokenAPIFY_TOKEN-
isAtHomeAPIFY_IS_AT_HOME-
defaultDatasetIdACTOR_DEFAULT_DATASET_ID'default'
defaultKeyValueStoreIdACTOR_DEFAULT_KEY_VALUE_STORE_ID'default'
defaultRequestQueueIdACTOR_DEFAULT_REQUEST_QUEUE_ID'default'

Advanced Configuration Options

KeyEnvironment VariableDefault Value
actorEventsWsUrlACTOR_EVENTS_WEBSOCKET_URL-
actorIdACTOR_ID-
actorRunIdACTOR_RUN_ID-
actorTaskIdACTOR_TASK_ID-
apiBaseUrlAPIFY_API_BASE_URL'https://api.apify.com'
containerPortACTOR_WEB_SERVER_PORT4321
containerUrlACTOR_WEB_SERVER_URL'http://localhost:4321'
inputKeyACTOR_INPUT_KEY'INPUT'
metamorphAfterSleepMillisAPIFY_METAMORPH_AFTER_SLEEP_MILLIS300e3
metaOriginAPIFY_META_ORIGIN-
proxyHostnameAPIFY_PROXY_HOSTNAME'proxy.apify.com'
proxyPasswordAPIFY_PROXY_PASSWORD-
proxyPortAPIFY_PROXY_PORT8000
proxyStatusUrlAPIFY_PROXY_STATUS_URL'http://proxy.apify.com'
userIdAPIFY_USER_ID-
userIsPayingAPIFY_USER_IS_PAYING-
xvfbAPIFY_XVFB-
standbyPortACTOR_STANDBY_PORT4321
standbyUrlACTOR_STANDBY_URL-
chromeExecutablePathAPIFY_CHROME_EXECUTABLE_PATH-
defaultBrowserPathAPIFY_DEFAULT_BROWSER_PATH-

Hierarchy

  • Configuration
    • Configuration

Index

Constructors

Properties

Methods

Constructors

externalconstructor

  • Creates new Configuration instance with provided options. Env vars will have precedence over those.

    Parameters

    • externaloptionaloptions: ConfigurationOptions

    Returns Configuration

Properties

externalreadonlyinheritedstorageManagers

storageManagers: Map<Constructor, StorageManager<IStorage>>

staticoptionalglobalConfig

globalConfig?: Configuration
@inheritDoc

Methods

get

  • get<T, U>(key, defaultValue): U
  • @inheritDoc

    Parameters

    • key: T
    • optionaldefaultValue: U

    Returns U

externalinheritedgetEventManager

  • getEventManager(): EventManager

  • Returns EventManager

set

  • set(key, value): void
  • @inheritDoc

    Parameters

    Returns void

externalinheriteduseEventManager

  • useEventManager(events): void

  • Parameters

    • externalevents: EventManager

    Returns void

externalinheriteduseStorageClient

  • useStorageClient(client): void

  • Parameters

    • externalclient: StorageClient

    Returns void

staticexternalinheritedgetEventManager

  • getEventManager(): EventManager

  • Gets default EventManager instance.

    Returns EventManager

staticgetGlobalConfig

  • @inheritDoc

    Returns Configuration

staticexternalinheritedgetStorageClient

  • getStorageClient(): StorageClient

  • Gets default StorageClient instance.

    Returns StorageClient

staticresetGlobalState

  • resetGlobalState(): void

  • Resets global configuration instance. The default instance holds configuration based on env vars, if we want to change them, we need to first reset the global state. Used mainly for testing purposes.

    Returns void

staticexternalinheritedset

  • set(key, value): void

  • Sets value for given option. Only affects the global Configuration instance, the value will not be propagated down to the env var. To reset a value, we can omit the value argument or pass undefined there.

    Parameters

    • externalkey: keyof ConfigurationOptions
    • externaloptionalvalue: any

    Returns void

staticexternalinheriteduseStorageClient

  • useStorageClient(client): void

  • Parameters

    • externalclient: StorageClient

    Returns void

Page Options