Configuration¶

Symbols Service configuration ¶

Gunicorn configuration:

GUNICORN_TIMEOUT¶

Parser:: str
Default:: “300”
Required:: No

Specifies the timeout value.

https://docs.gunicorn.org/en/stable/settings.html#timeout

Used in bin/run_web.sh.

GUNICORN_WORKERS¶

Parser:: str
Default:: “1”
Required:: No

Specifies the number of gunicorn workers.

You should set it to (2 x $num_cores) + 1.

https://docs.gunicorn.org/en/stable/settings.html#workers

http://docs.gunicorn.org/en/stable/design.html#how-many-workers

Used in bin/run_web.sh.

Webapp configuration:

Configuration¶

Configuration summary:

Setting	Parser	Required?
`LOCAL_DEV_ENV`	bool
`TEST_ENV`	bool
`TOOL_ENV`	bool
`SENTRY_DSN`	str
`HOSTNAME`	str
`LOGGING_DEFAULT_LEVEL`	str
`STATSD_HOST`	str
`STATSD_PORT`	int
`STATSD_NAMESPACE`	str
`FRONTEND_ROOT`	str
`STATIC_ROOT`	str
`OIDC_RP_CLIENT_ID`	str	Yes
`OIDC_RP_CLIENT_SECRET`	str	Yes
`OIDC_OP_AUTHORIZATION_ENDPOINT`	str	Yes
`OIDC_OP_TOKEN_ENDPOINT`	str	Yes
`OIDC_OP_USER_ENDPOINT`	str	Yes
`OIDC_VERIFY_SSL`	bool
`ENABLE_TOKENS_AUTHENTICATION`	bool
`TOKENS_DEFAULT_EXPIRATION_DAYS`	int
`REDIS_URL`	str	Yes
`CLOUD_SERVICE_PROVIDER`	str
`AWS_ACCESS_KEY_ID`	str
`AWS_SECRET_ACCESS_KEY`	str
`AWS_DEFAULT_REGION`	str
`MEMOIZE_KEY_EXISTING_SIZE_SECONDS`	int
`UPLOAD_FILE_UPLOAD_MAX_WORKERS`	int
`UPLOAD_TEMPDIR`	str
`UPLOAD_TEMPDIR_ORPHANS_CUTOFF`	int
`ALLOW_UPLOAD_BY_ANY_DOMAIN`	bool
`SYNCHRONOUS_UPLOAD_FILE_UPLOAD`	bool
`SECRET_KEY`	str	Yes
`DEBUG`	bool
`ALLOWED_HOSTS`	ListOf(str)
`DATABASE_URL`	dj_database_url.parse	Yes
`CONN_MAX_AGE`	int
`SESSION_COOKIE_AGE`	int
`SYMBOL_URLS`	ListOf(str)	Yes
`UPLOAD_DEFAULT_URL`	str	Yes
`UPLOAD_TRY_SYMBOLS_URL`	str	Yes
`SYMBOL_FILE_PREFIX`	str
`COMPRESS_EXTENSIONS`	ListOf(str)
`MIME_OVERRIDES`	dict_parser
`DISALLOWED_SYMBOLS_SNIPPETS`	ListOf(str)
`UPLOAD_REATTEMPT_LIMIT_SECONDS`	int
`ALLOW_UPLOAD_BY_DOWNLOAD_DOMAINS`	ListOf(str)
`DOWNLOAD_FILE_EXTENSIONS_ALLOWED`	ListOf(str)

Configuration options:

LOCAL_DEV_ENV¶

Parser:: bool
Default:: “false”
Required:: No

Set to true if you’re running in a local dev environment; false otherwise

TEST_ENV¶

Parser:: bool
Default:: “false”
Required:: No

Set to true if you’re running tests; false otherwise.

TOOL_ENV¶

Parser:: bool
Default:: “false”
Required:: No

Set to true if you’re running manage.py in a tool context. For example, for collectstatic.

SENTRY_DSN¶

Parser:: str
Default:: “”
Required:: No

Sentry DSN or empty string

HOSTNAME¶

Parser:: str
Default:: “socket.gethostname()”
Required:: No

Unique identifier for the host that is running Tecken. This is used in logging and metrics. The default is socket.gethostname().

LOGGING_DEFAULT_LEVEL¶

Parser:: str
Default:: “INFO”
Required:: No

Default level for logging. Should be one of INFO, DEBUG, WARNING, ERROR.

STATSD_HOST¶

Parser:: str
Default:: “localhost”
Required:: No

statsd host.

STATSD_PORT¶

Parser:: int
Default:: “8125”
Required:: No

statsd port.

STATSD_NAMESPACE¶

Parser:: str
Default:: “”
Required:: No

Namespace for statsd keys.

FRONTEND_ROOT¶

Parser:: str
Default:: “os.path.join(BASE_DIR, “frontend/build/”)”
Required:: No

Root directory for frontend files like index.html

STATIC_ROOT¶

Parser:: str
Default:: “os.path.join(BASE_DIR, “frontend/build/static/”)”
Required:: No

Root directory for static files.

OIDC_RP_CLIENT_ID¶

Parser:: str
Required:: Yes

OIDC RP client id.

OIDC_RP_CLIENT_SECRET¶

Parser:: str
Required:: Yes

OIDC RP client secret.

OIDC_OP_AUTHORIZATION_ENDPOINT¶

Parser:: str
Required:: Yes

OIDC OP authorization endpoint.

OIDC_OP_TOKEN_ENDPOINT¶

Parser:: str
Required:: Yes

OIDC OP token endpoint.

OIDC_OP_USER_ENDPOINT¶

Parser:: str
Required:: Yes

OIDC OP user endpoint.

OIDC_VERIFY_SSL¶

Parser:: bool
Default:: “true”
Required:: No

Whether or not to verify SSL. This should always be True unless in a local dev environment.

ENABLE_TOKENS_AUTHENTICATION¶

Parser:: bool
Default:: “true”
Required:: No

True if API token authentication is enabled; false otherwise.

TOKENS_DEFAULT_EXPIRATION_DAYS¶

Parser:: int
Default:: “365”
Required:: No

Default expiration in days for tokens.

REDIS_URL¶

Parser:: str
Required:: Yes

URL for Redis.

CLOUD_SERVICE_PROVIDER¶

Parser:: str
Default:: “AWS”
Required:: No

The cloud service provider Tecken is using. Either AWS or GCP.

AWS_ACCESS_KEY_ID¶

Parser:: str
Default:: “”
Required:: No

AWS access key id.

AWS_SECRET_ACCESS_KEY¶

Parser:: str
Default:: “”
Required:: No

AWS secret access key.

AWS_DEFAULT_REGION¶

Parser:: str
Default:: “”
Required:: No

AWS default region.

MEMOIZE_KEY_EXISTING_SIZE_SECONDS¶

Parser:: int
Default:: “str(60 * 60 * 24)”
Required:: No

When we ask S3 for the size (if it exists) of a symbol already in S3 this can be cached. This value determines how long we do that caching.

UPLOAD_FILE_UPLOAD_MAX_WORKERS¶

Parser:: int
Default:: “0”
Required:: No

When we upload a .zip file, we iterate over the content and for each file within (that isn’t immediately ignorable) we kick off a function which figures out what (and how) to process the file. That function involves doing a S3 GET (technically ListObjectsV2), (possible) gzipping the payload and (possibly) a S3 PUT. All of these function calls get put in a concurrent.futures.ThreadPoolExecutor pool. This setting is about how many of these to start, max.

UPLOAD_TEMPDIR¶

Parser:: str
Default:: “/tmp/uploads”
Required:: No

The directory to use as a workspace for handling symbol uploads.

UPLOAD_TEMPDIR_ORPHANS_CUTOFF¶

Parser:: int
Default:: “15”
Required:: No

Time in minutes before we consider a file to have been orphaned and should be deleted.

ALLOW_UPLOAD_BY_ANY_DOMAIN¶

Parser:: bool
Default:: “false”
Required:: No

When doing local development, especially load testing, it’s sometimes useful to be able to bypass all URL checks for Upload by Download.

SYNCHRONOUS_UPLOAD_FILE_UPLOAD¶

Parser:: bool
Default:: “false”
Required:: No

This is only really meant for the sake of being overrideable by other setting classes; in particular when running tests.

SECRET_KEY¶

Parser:: str
Required:: Yes

Django’s secret key for signing things.

DEBUG¶

Parser:: bool
Default:: “false”
Required:: No

Whether or not to enable debug mode. Don’t set this to True in server environments

ALLOWED_HOSTS¶

Parser:: ListOf(str)
Default:: “”
Required:: No

Comma-delimited list of strings of host/domain names for this site.

DATABASE_URL¶

Parser:: dj_database_url.parse
Required:: Yes

The database_url to use. This gets parsed into DATABASES configuration.

CONN_MAX_AGE¶

Parser:: int
Default:: “60”
Required:: No

Maximum age in minutes for connections.

SESSION_COOKIE_AGE¶

Parser:: int
Default:: “str(60 * 60 * 24 * 365)”
Required:: No

Age in seconds for cookies. Keep it quite short because we don’t have a practical way to do OIDC ID token renewal for this AJAX and curl heavy app.

SYMBOL_URLS¶

Parser:: ListOf(str)
Required:: Yes

Comma-separated list of urls for symbol downloads.

Lookups are performed in list order.

UPLOAD_DEFAULT_URL¶

Parser:: str
Required:: Yes

The default url to use for symbol uploads. This must be an item in SYMBOL_URLS.

UPLOAD_TRY_SYMBOLS_URL¶

Parser:: str
Required:: Yes

When an upload comes in with symbols from a Try build, these symbols mustn’t be uploaded with the regular symbols.

You could set this to UPLOAD_DEFAULT_URL with a ‘/try’ prefix.

For example:

UPLOAD_DEFAULT_URL=http://s3.example.com/publicbucket/
UPLOAD_TRY_SYMBOLS_URL=http://s3.example.com/publicbucket/try/

SYMBOL_FILE_PREFIX¶

Parser:: str
Default:: “v1”
Required:: No

Prefix in the bucket for all symbol files. This allows us to change the file path template.

COMPRESS_EXTENSIONS¶

Parser:: ListOf(str)
Default:: “sym”
Required:: No

During upload, for each file in the archive, if the extension matches this list, the file gets gzip compressed before uploading.

MIME_OVERRIDES¶

Parser:: dict_parser
Default:: “{“sym”:”text/plain”}”
Required:: No

For specific file uploads, override the mimetype.

For .sym files, for example, if S3 knows them as ‘text/plain’ they become really handy to open in a browser and view directly.

DISALLOWED_SYMBOLS_SNIPPETS¶

Parser:: ListOf(str)
Default:: “qcom/proprietary”
Required:: No

Individual strings that can’t be allowed in any of the lines in the content of a symbols archive file.

UPLOAD_REATTEMPT_LIMIT_SECONDS¶

Parser:: int
Default:: “str(60 * 60 * 12)”
Required:: No

Every time we do a symbol upload, we also take a look to see if there are incomplete uploads that could have failed due to some unlucky temporary glitch.

When we do the reattempt, we need to wait sufficiently long because the upload might just be incomplete because it’s in the queue, not because it failed.

ALLOW_UPLOAD_BY_DOWNLOAD_DOMAINS¶

Parser:: ListOf(str)
Default:: “queue.taskcluster.net,firefox-ci-tc.services.mozilla.com,stage.taskcluster.nonprod.cloudops.mozgcp.net”
Required:: No

When you “upload by download”, the URL’s domain needs to be in this allow list. This is to double-check that we don’t allow downloads from domains we don’t fully trust.

DOWNLOAD_FILE_EXTENSIONS_ALLOWED¶

Parser:: ListOf(str)
Default:: “.sym,.dl_,.ex_,.pd_,.dbg.gz,.tar.bz2”
Required:: No

A list of file extensions that if a file is NOT one of these extensions we can immediately return 404 and not bother to process for anything else.

It’s case sensitive and has to be lower case. As a way to get marginal optimization of this, make sure ‘.sym’ is first in the list since it’s the most common.

Configuration¶

Symbols Service configuration¶

Symbols Service configuration ¶