Seer

Seer is a service that provides AI capabilities to Sentry by running inference on Sentry issues and providing user insights.

📣 Seer is currently in early development and not yet compatible with self-hosted Sentry instances. Stay tuned for updates!

Setup

These instructions require access to internal Sentry resources and are intended for internal Sentry employees.

Prerequisites

1. Install direnv or a similar tool
2. Install pyenv and configure Python 3.11
3. Install Docker
4. Install Google Cloud SDK

Environment Setup

1. Clone the repository and navigate to the project root
2. Run direnv allow to set up the Python environment
3. Create a .env file based on .env.example and set the required values
4. (Optional) Add SENTRY_AUTH_TOKEN=<your token> to your .env file

Model Artifacts

Download model artifacts:

gsutil cp -r gs://sentry-ml/seer/models ./models

Running Seer

1. Start the development environment:

   make dev

2. If you encounter database errors, run:

   make update

Integrating with Local Sentry

1. Expose port 9091 in your local Sentry configuration

2. Add the following to ~/.sentry/sentry.conf.py:

   SEER_RPC_SHARED_SECRET = ["seers-also-very-long-value-haha"]
   SENTRY_FEATURES['projects:ai-autofix'] = True
   SENTRY_FEATURES['organizations:issue-details-autofix-ui'] = True

3. For local development, you may need to bypass certain checks in the Sentry codebase

4. Restart both Sentry and Seer

Note

Set NO_SENTRY_INTEGRATION=1 in .env to ignore Local Sentry Integration

Development Commands

• Apply database migrations: make update
• Create new migrations: make migration
• Run type checker: make mypy
• Run tests: make test
• Open a shell: make shell

Reset Development Environment

To start fresh:

bash
docker compose down --volumes
make update && make dev

Langfuse Integration

To enable Langfuse tracing, set these environment variables:

LANGFUSE_SECRET_KEY=...
LANGFUSE_PUBLIC_KEY=...
LANGFUSE_HOST=...

Autofix

Autofix is an AI agent that identifies root causes of Sentry issues and suggests fixes.

Running Evaluations

Send a POST request to /v1/automation/autofix/evaluations/start with the following JSON body:

{
"dataset_name": "string", // Name of the dataset to run on (currently only internal datasets available)
"run_name": "string", // Custom name for your evaluation run
"run_description": "string", // Description of your evaluation run
"run_type": "full | root_cause | execution", // Type of evaluation to perform
"test": boolean, // Set to true to run on a single item (for testing)
"random_for_test": boolean, // Set to true to use a random item when testing (requires "test": true)
"run_on_item_id": "string", // Specific item ID to run on (optional)
"n_runs_per_item": int // Number of runs to perform per item (optional, default 1)
}

Note: Currently, only internal datasets are available.

Staging Sandbox

It is possible to run and deploy seer to a sandbox staging environment. An example of such a deployment is in this PR.

To get started, use the #proj-tf-sandbox channel and request direction or help on scaffolding a new sandbox in the sandbox repo.

You then can use the iap, and seer-staging modules to scaffold a public load balancer pointing to a compute running the docker-compose.staging.yml file.

After scaffolding your environment, you'll want to set your SBX_PROJECT environment variable in your .env file, and run make push-staging to submit a cloud build for your image.

!!!!NOTE!!!! The staging cloud build uses your current local environment to build the image, not CI, which means it will use all your src files and your local .env file to configure the image that will be hosted in your sandbox. Make sure you don't accidentally include any sensitive personal files in your source tree before using this.

Each time you push with make push-staging there will be a period of time while the VM polls and unpacks the new image before it is loaded. If you have a SENTRY_DSN and SENTRY_ENVIRONMENT set, a release will be created by the push, allowing you to track when the server has loaded that release version.