Testing FastAPI and Async SQLAlchemy After Django
In Django you inherit from TestCase, use self.client, and everything works.
The test database gets created, each test runs in a transaction that rolls back,
and you never think about it again.
FastAPI gives you none of that. You build all of it in conftest.py, and the
parts that bite have nothing to do with your app code. They are about event
loops and about which database your migrations think they are talking to.
Here is what I ended up with on CodifyLive.
Import order comes first
Settings() is built the moment app.core.config is imported, and app.db
creates its engine at import time too. So the test database has to be chosen
before any app.* import runs.
load_dotenv()
os.environ["DATABASE_URL"] = os.environ["TEST_DATABASE_URL"]
import pytest
from app.main import appThat one line also points Alembic at the test database, because migrations/env.py
reads settings.DATABASE_URL. Setting sqlalchemy.url on the Alembic Config
object does nothing, env.py overwrites it.
Django reads DATABASES lazily and swaps in a test database for you. Nothing
here is lazy.
Run migrations, not create_all
@pytest.fixture(scope="session", autouse=True)
def apply_migrations():
cfg = Config("alembic.ini")
command.upgrade(cfg, "head")
yield
command.downgrade(cfg, "base")The fixture has to be sync. Alembic's async env.py calls asyncio.run(), which
throws if a loop is already running.
Use migrations rather than create_all so the tests exercise the real schema,
check constraints and partial indexes included. It also means a broken migration
fails your suite, which is the point.
Use Postgres
SQLite in memory is tempting and wrong. gen_random_uuid(), the postgres UUID
type, and partial indexes do not exist there. You would be testing a schema you
never ship.
NullPool, or you get a loop error
create_async_engine(settings.DATABASE_URL, poolclass=NullPool)pytest-asyncio gives every test its own event loop. asyncpg binds a connection to the loop that opened it, so a pooled connection reused by the next test blows up:
RuntimeError: ... attached to a different loopIt only shows up once two tests both touch the database, which is a fun way to find out.
Rollback needs create_savepoint
async with engine.connect() as connection:
transaction = await connection.begin()
session_factory = async_sessionmaker(
bind=connection,
expire_on_commit=False,
join_transaction_mode="create_savepoint",
)
async with session_factory() as session:
yield session
await transaction.rollback()This is Django's rollback behaviour, rebuilt by hand.
join_transaction_mode="create_savepoint" is the part people miss. Your handlers
call await db.commit(), and a commit on a session bound to an outer transaction
ends that transaction. The rollback then does nothing and rows leak between
tests. Passes alone, fails in the suite.
Override dependencies instead of patching
@pytest.fixture
async def client(db_session):
async def _override_get_db():
yield db_session
app.dependency_overrides[get_db] = _override_get_db
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as ac:
yield ac
app.dependency_overrides.clear()That clear() is not optional. app is a module-level singleton, so a leaked
override quietly corrupts every test after it.
Use AsyncClient with ASGITransport, not TestClient. TestClient runs the
app in its own thread with its own loop, which fights the asyncpg connections
your fixtures created.
The tests themselves
Once the plumbing is right, the tests read fine:
class TestLogin:
async def test_unknown_email_returns_same_401_as_wrong_password(self, client, user_password):
response = await client.post(
"/auth/login",
json={"email": "nobody@mike.dev", "password": "SuperSecretPassword123@"}
)
assert response.status_code == 401
assert response.json()["detail"] == "Invalid credentials."Two things that cost me time here.
Cookie attributes are not on response.cookies. That returns the value string.
To assert on httponly or path you parse the header:
jar = SimpleCookie()
jar.load(response.headers["set-cookie"])
cookie = jar["refresh"]
assert cookie["httponly"]
assert cookie["path"] == "/auth"And two JWTs minted in the same second are byte identical, iat and exp only
go down to seconds. Do not assert a refreshed token differs from the old one.
Decode it and assert on sub and type instead.
It found a real bug
My refresh endpoint revokes every token for a user when a revoked one gets presented again, that is the reuse detection. The test replayed an old cookie and got a 500 instead of a 401:
AttributeError: 'function' object has no attribute 'all'refresh_token_rows.scalars.all(), missing the parentheses. The mass revocation
had never run once in production. One character, in the one code path that only
executes when someone is actually attacking you.
That is the argument for testing this stuff. Not coverage, just the paths you cannot trigger by clicking around.
Worth it
The setup is maybe 80 lines you write once. After that it behaves like Django, except you know exactly what every piece does.
Full code, conftest and the auth tests, is here: github.com/fulanii/codify-live-backend.