Inventory conventionally named Python tests

Before you begin

Python 3.12 or newer

What this script changesReads Python source and writes a new JSON result without changing or running analyzed code. Maximum 500 files and 2 MiB per file. Static findings are review aids; dynamic behavior and runtime imports are outside scope.

Set up your example

Save the complete code as python-test-discovery.py in a working folder.
Save the provided sample files in that same folder; run commands from that folder.

Sample files in this download:

sample.py

Download the ZIP, extract it, and open a terminal inside the extracted folder. The source file is python-test-discovery.py.

Run the script

python "python-test-discovery.py" --source "sample.py" --output "analysis.json"

If your system uses python3 instead of python, use that command for Python examples. Keep paths containing spaces in quotes. Required inputs and optional settings are described in the code and setup steps.

What to expect

Locate test_ functions and Test classes without running tests. test_add is found with one assert statement.

The complete script

#!/usr/bin/env python3
'Inventory conventionally named Python tests'
import argparse
import json
from pathlib import Path

def write_json(path, value):
    data = json.dumps(value, indent=2, ensure_ascii=False, allow_nan=False) + "\n"
    with Path(path).open("x", encoding="utf-8", newline="") as handle:
        handle.write(data)

import ast
import tokenize
import io
import doctest
from collections import Counter, defaultdict

FUNCTIONS=(ast.FunctionDef,ast.AsyncFunctionDef)
SCOPES=(ast.FunctionDef,ast.AsyncFunctionDef,ast.ClassDef,ast.Lambda)
def functions(tree):
    return [n for n in ast.walk(tree) if isinstance(n,FUNCTIONS)]
def local_nodes(node):
    yield node
    for child in ast.iter_child_nodes(node):
        if not isinstance(child,SCOPES):
            yield from local_nodes(child)
def name(node):
    try: return ast.unparse(node)
    except (ValueError,TypeError): return type(node).__name__
def targets(node):
    return [n.id for n in ast.walk(node) if isinstance(n,ast.Name) and isinstance(n.ctx,ast.Store)]
def sources(value):
    root=Path(value)
    if root.is_file(): return [root]
    if not root.is_dir(): raise ValueError("--source must be a Python file or directory.")
    excluded={".git",".venv","venv","node_modules","__pycache__"}
    files=sorted(p for p in root.rglob("*.py") if p.is_file() and not any(x in excluded or x.startswith(".") for x in p.relative_to(root).parts))
    if len(files)>500: raise ValueError("At most 500 Python files per run.")
    return files


def analyze(tree,text):
    result=[]
    for n in ast.walk(tree):
        if isinstance(n,FUNCTIONS) and n.name.startswith("test_"):
            result.append({"kind":"function","name":n.name,"line":n.lineno,"assertions":sum(isinstance(x,ast.Assert) for x in local_nodes(n))})
        elif isinstance(n,ast.ClassDef) and n.name.startswith("Test"):
            result.append({"kind":"class","name":n.name,"line":n.lineno})
    return result

def main():
    parser = argparse.ArgumentParser(description='Inventory conventionally named Python tests')
    parser.add_argument("--source",required=True,help="Python file or source directory")
    parser.add_argument("--output",required=True)
    args = parser.parse_args()
    reports=[]
    for path in sources(args.source):
        try:
            if path.stat().st_size>2*1024*1024: raise ValueError("Source exceeds 2 MiB.")
            with tokenize.open(path) as handle: text=handle.read()
            tree=ast.parse(text,filename=str(path))
            reports.append({"path":str(path),"result":analyze(tree,text)})
        except (OSError,SyntaxError,UnicodeError,ValueError) as exc:
            reports.append({"path":str(path),"error":str(exc)})
    write_json(args.output,reports)

if __name__ == "__main__":
    try:
        main()
    except (OSError, ValueError, KeyError, TypeError, ArithmeticError, IndexError, LookupError) as exc:
        raise SystemExit(str(exc))

Open source as plain text · Download source and sample files

How it works

The program parses source syntax without importing or executing it; per-file parse failures become error records.
Naming conventions are only discovery hints; fixtures, decorators, and custom test-runner collection rules are not evaluated.
test_add is found with one assert statement.

Make it your own

--source

Use --source with a Python file or directory; directory mode skips common dependency and hidden folders.

Adapt the processing

Edit analyze's prefix conditions for a team's naming convention.

Change one setting at a time, run it again on sample data, and compare the output. The customization guide walks through this process.

If something goes wrong

A runner collects different tests.

This is a static inventory, not a replacement for runner-specific collection.

The destination already exists.

Choose a new --output path. The script deliberately refuses to replace an existing result.

Reference & next steps

These official references explain the underlying tools. The Scriptage example and guide are original instructional content. See verification notes for what has been checked and the limits of those checks.

Copies a template for your own notes. It does not send anything.