Finalizing Database Stored Settings

testgrinder service will be discontinued on January 31, 2025

testgrinder SMPC was discontinued as of February 1, 2024

You complete the initial deployment by setting several database stored settings, some of which are required and some are optional. The table below lists the settings that you need to focus on during the initial deployment. See Database Stored Settings for how to apply the changes.

Setting	Change requried?	Description
aws_vpc_subnet_ids	Yes	An array of subnet ids in which tg-app can launch tg-bots
max_aws_instances	No default: 20	Hard limit to how many EC2 instances tg-apps are allowed to launch. Raise to a higher number if required.
ec2_instance_key_name	No default: none	If specified, tg-bots will be launched with this key assigned
media_s3_bucket	Yes	Name of the S3 bucket where testgrinder should store captured screenshots and videos
user_files_s3_bucket	Yes	Name of the S3 bucket where user files attached to projects should be stored
tg_api_url	Yes	URL that tg-bots will use to connect to tg-app. Should be 'TG_APP_PRIVATE_HOST:3000/ra' where TG_APP_PRIVATE_HOST is either tg-app DNS name on the private network, or, if tg-apps are managed by an Auto Scaling Group, DNS name of the internal Load Balancer. tg-bot will not be able to communicate with tg-app if you use public tg-app DNS name here.
aws_runner_image_id	Yes	ID of the AMI to be used for launching tg-bots
aws_runner_security_groups	Yes	Array of security groups (their IDs, not names) to be assigned to launched tg-bots. List here the security group created in Security Groups for tg-bot.
max_runners	No default: 10	Limit of how many tg-bots may run at the same time.
ec2_runner_instance_type	No default: c6a.large	EC2 Instance Type to use when launching tg-bots. It needs to have x86_64 architecture, at least 4GB of RAM, and preferably be compute optimized.
aws_runner_iam_role	Yes	Set to the role created for tg-bot in IAM Roles

Generate Settings Script

To change database settings, first export the script with the current settings as described below. Then make changes to the script and run it to apply the changes.

$ ssh ubuntu@my-tg-app
$ sudo su - testgrinder
$ cd tg-app

• substitute my-tg-app with the public domain of your tg-app instance

• testgrinder is deployed under ubuntu user testgrinder

When running the export-settings script, supply the name of the output script as an argument:

$ script/export-settings my_settings

Example script for setting database stored settings

This is a shortened list of the generated my_settings script listing only the settings that are important during the initial deployment. Make sure to substitute your own values before running. This is not an exhaustive list of available settings. See the full list in Database Stored Settings

#!/usr/bin/env -S rails runner

require 'scripts/settings'

## AWS

# array of AWS VPC subnet IDs in which tg-app can spawn EC2 instances
set aws_vpc_subnet_ids: ["subnet-45298347536", "subnet-0434985435"]

# A hard limit on the total number of AWS instances. testgrinder will not launch more if this limit is reached.
set max_aws_instances: 50

# if specified, any launched EC2 instance (bot/gateway) will be configured to authorize ssh access for this key
set ec2_instance_key_name: "MyKey"


## Storage

# S3 bucket for storing screenshots and videos
set media_s3_bucket: "my_tg_media_bucket"

# S3 bucket for storing user files attached to projects
set user_files_s3_bucket: "my_tg_user_files_bucket"


## Bots

# URL bots should use to communicate with tg-app API, e.g. my_testgrinder:3000/ra
set tg_api_url: "my_testgrinder:3000/ra"

# AMI image ID used to launch bots
set aws_runner_image_id: "ami-04f2ac1ea3bae6d95"

# Security groups to assign to a bot on launch
set aws_runner_security_groups: ["sg-0f294d9d2d25bb9c8"]

# A hard limit on the total number of bots
set max_runners: 20

# EC2 instance type to be used for launching bots
set ec2_runner_instance_type: "c6a.large"

# bot AWS IAM role
set aws_runner_iam_role: "tg-eval-bot"

Apply the changes by logging in to the tg-app server via ssh as user ubuntu and then running the script as user testgrinder, assuming the script is stored in /home/testgrinder/tg-app/my_settings:

$ ssh ubuntu@my-tg-app
$ sudo su - testgrinder
$ cd tg-app
$ chmod +x my_settings
$ ./my_settings
Changed aws_vpc_subnet_ids: from nil => ["subnet-45298347536", "subnet-0434985435"]
Changed max_aws_instances: from 20 => 50
Changed ec2_instance_key_name: from nil => "MyKey"
Changed media_s3_bucket: from nil => "my_tg_media_bucket"
Changed user_files_s3_bucket: from nil => "my_tg_user_files_bucket"
Changed tg_api_url: from nil => "my_testgrinder:3000/ra"
Changed aws_runner_image_id: from nil => "ami-04f2ac1ea3bae6d95"
Changed aws_runner_security_groups: from nil => ["sg-0f294d9d2d25bb9c8"]
Changed max_runners: from 10 => 20
Unchanged ec2_runner_instance_type: "c6a.large"
Changed aws_runner_iam_role: from nil => "tg-eval-bot"