dobbe

dobbe test gen

Generate tests for untested code paths using an agentic analyze-generate-verify loop.

Synopsis

dobbe test gen [OPTIONS]

Description

test gen orchestrates multiple AI agents in an iterative pipeline to:

1. Analyze the codebase for coverage gaps (untested files, functions, branches)
2. Generate test files targeting those gaps
3. Run the tests and verify they pass
4. Retry with error feedback if tests fail
5. Commit passing tests and optionally create a PR

The pipeline runs up to --max-iterations generate-verify cycles before reporting results. It supports pytest, jest, go, and mocha frameworks, with automatic detection when --framework is not specified.

Pipeline Flowchart

+---------------------------------------------------+
|  dobbe test gen [--repo org/repo]                  |
|  (auto-detects from CWD if no flags given)         |
+-------------------------+-------------------------+
                          |
                          v
+---------------------------------------------------+
|  1. DISCOVER                                       |
|  - Auto-detect repo from CWD git remote            |
|  - Discover available MCP servers                  |
|  - Resolve repo: configured paths -> common        |
|    dirs -> CWD remote match -> clone to tempdir    |
+-------------------------+-------------------------+
                          |
                          v
+---------------------------------------------------+
|  2. BRANCH SETUP                                   |
|  - Create feature branch from --base               |
|  - Branch name: dobbe/add-tests-<org>-<repo>       |
|  (skipped in --dry-run mode)                       |
+-------------------------+-------------------------+
                          |
                          v
+---------------------------------------------------+
|  3. ANALYZE AGENT                                  |
|  - Pre-discover source and test files              |
|  - Identify untested code paths                    |
|  - Detect test framework if not specified          |
|  - Focus on --file or --pr changes if given        |
|  - Output: coverage gaps list                      |
|  Tools: [Bash, Read, Grep, Glob] + MCP             |
+-------------------------+-------------------------+
                          |
                    +-----+------+
                    |  Dry run?  |
                    +-----+------+
                  yes |       | no
                      v       v
               +---------+  +----------------------------+
               | Report  |  |  4. GENERATE AGENT          |<--------+
               | gaps    |  |  - Read coverage gaps       |         |
               | + exit  |  |  - Write test files         |         |
               +---------+  |  - Target untested paths    |         |
                            |  - Apply error feedback     |         |
                            |    from prior iteration     |         |
                            |  Tools: [Bash, Read, Grep,  |         |
                            |   Glob, Edit, Write] + MCP  |         |
                            +-------------+--------------+         |
                                          |                        |
                                          v                        |
                            +----------------------------+         |
                            |  5. VERIFY AGENT            |         |
                            |  - Run generated tests      |         |
                            |  - Check for failures       |         |
                            |  - Report pass/fail + output|         |
                            |  Tools: [Bash, Read,        |         |
                            |   Grep, Glob]               |         |
                            +-------------+--------------+         |
                                          |                        |
                                    +-----+-----+                  |
                                    |  Passed?  |                  |
                                    +-----+-----+                  |
                                  yes |       | no                 |
                                      |       v                    |
                                      |  +-------------+           |
                                      |  | Iterations  |           |
                                      |  | remaining?  |           |
                                      |  +------+------+           |
                                      |  yes |      | no          |
                                      |      v      |             |
                                      |  +--------+ |             |
                                      |  | RETRY  | |             |
                                      |  | Feed   | |             |
                                      |  | back   +-+-------------+
                                      |  | errors | |
                                      |  +--------+ |
                                      |             |
                                      v             v
                            +----------------------------+
                            |  6. COMMIT + PR             |
                            |  - Commit generated tests   |
                            |  - Push branch              |
                            |  - Create PR via gh CLI     |
                            |  (skipped if --no-pr or     |
                            |   not converged)            |
                            +----------------------------+

Options

Option	Type	Default	Description
--repo / -r	str	auto-detect	Repository to generate tests for (org/repo). Auto-detected from current directory if omitted.
--file / -f	str	-	Target a specific file for test generation
--pr	int	-	Generate tests for changes in a specific PR number
--framework	str	auto-detect	Test framework: pytest, jest, go, mocha
--max-iterations	int	3	Max generate-verify iterations (1-10)
--create-pr / --no-pr	bool	True	Create PR with generated tests on convergence
--dry-run	bool	False	Analyze only - identify coverage gaps without generating tests
--coverage-threshold	float	0.0	Minimum coverage target (percent, 0-100)
--base	str	auto-detect	Base branch to branch from
--format	str	table	Output format: table, json, markdown
--quiet / -q	bool	False	Suppress progress output

When run from inside a git repository with no --repo flag, the repo is auto-detected from the current directory’s git remote URL. The --framework is auto-detected from the analyze agent’s response when not specified.

Agent Tool Access

Each pipeline agent has scoped tool permissions:

Agent	Tools	Why
Analyze	Bash, Read, Grep, Glob + MCP	Read-only analysis of source and test files
Generate	Bash, Read, Grep, Glob, Edit, Write + MCP	Needs to create and write test files
Verify	Bash, Read, Grep, Glob	Run tests and read output, no edits allowed

MCP tools (e.g., mcp__github__*) are added when available for the analyze and generate agents.

Pipeline Stages

Stage 1: Discovery

Discovers available MCP servers from ~/.claude/settings.json and resolves the repository location:

1. If no --repo flag, auto-detect from current directory’s git remote URL
2. Check configured local_paths in ~/.dobbe/config.toml
3. Scan common project directories (~/projects, ~/code, ~/src, etc.)
4. Check if current directory’s git remote matches the --repo slug
5. Fall back to shallow clone (git clone --depth 1) into a temp directory

Stage 2: Branch Setup

Creates a feature branch from the base branch. Default branch name follows the pattern dobbe/add-tests-<org>-<repo>. The base branch is auto-detected if --base is not specified. This stage is skipped entirely in --dry-run mode.

Stage 3: Analyze

The analyze agent identifies untested code paths:

• Pre-discovers source and test files using framework-aware glob patterns
• Compares source files against existing test files to find untested modules
• When --file is specified, focuses specifically on that file and its dependencies
• When --pr is specified, runs gh pr diff to identify changed files and prioritizes coverage for those changes
• Detects the test framework from the codebase if --framework was not set
• Returns a structured list of coverage gaps (file path, function name, line range, reason)

Stage 4: Generate

The generate agent reads the coverage gaps and writes test files:

• Creates test files following the project’s existing test conventions
• Targets the specific untested functions and code paths from the analysis
• On retry iterations, incorporates feedback from the verify agent about what failed
• Uses session continuations to maintain context across iterations

Stage 5: Verify

The verify agent runs the generated tests:

• Executes the test suite using the detected framework
• Reports pass/fail status and captures test output
• If tests fail, extracts error messages as feedback for the next generate iteration

If verification fails and iterations remain, the pipeline feeds the error details back to the generate agent and starts the next iteration. If all iterations are exhausted without passing, the pipeline reports non-convergence.

Stage 6: Commit and PR

If the pipeline converged (tests pass) and --create-pr is enabled:

• Commits all generated test files with message dobbe: add generated tests
• Pushes the feature branch
• Creates a PR via gh pr create with a summary of tests generated, files covered, framework, and iteration count

Session Saving

Each pipeline run saves its output to the session store, enabling cross-command context. For example, a previous dobbe vuln scan or dobbe vuln resolve session for the same repo is loaded and provided as additional context to the analyze agent.

Convergence Logic

The pipeline converges (succeeds) when:

1. Verify passes - all generated tests pass
2. No coverage gaps - the analyze agent found nothing to test (unlikely but possible)

If the pipeline exhausts all iterations without converging, the report still includes all generated tests and coverage gaps with a non-convergence status. No PR is created in this case.

Examples

Generate tests for the current repository (auto-detect)

cd ~/projects/api
dobbe test gen

Generate tests for a specific repository

dobbe test gen --repo acme/web-app

Target a specific file

dobbe test gen --repo acme/web-app --file src/auth/oauth.py

Generate tests for PR changes

dobbe test gen --repo acme/web-app --pr 42

Dry run - identify coverage gaps without generating tests

dobbe test gen --repo acme/web-app --dry-run

Specify the test framework

dobbe test gen --repo acme/web-app --framework jest

Increase iteration count for thorny codebases

dobbe test gen --repo acme/web-app --max-iterations 5

Generate without creating a PR

dobbe test gen --repo acme/web-app --no-pr

JSON output for CI integration

dobbe test gen --repo acme/web-app --format json --quiet

Custom base branch

dobbe test gen --repo acme/web-app --base develop

See Also

• vuln resolve - similar agentic pipeline for vulnerability fixing
• Configuration reference - pipeline-related config options

This site is open source. Improve this page.